DIGITAL FILTERS 
AND 


SIGNAL PROCESSIN 


Edited by 
Fausto Pedro Garcia Marquez 
Noor Zaman 


DIGITAL FILTERS AND 
SIGNAL PROCESSING 


Edited by Fausto Pedro Garcia Marquez and 
Noor Zaman 


Digital Filters and Signal Processing 
http://dx.doi.org/10.5772/45654 
Edited by Fausto Pedro Garcia Marquez and Noor Zaman 


Contributors 


Barmak Honarvar Shakibaei Asli, Raveendran Paramesran, Alexey V. Mokeev, Jan Peter Hessling, Masayuki Kawamata, 
Shunsuke Yamaki, Masahide Abe, Radu Matei, Daniela Matei, Fumio Itami, Behrouz Nowrouzian, Seyyed Ali Hashemi, 
Fausto Pedro Garcia Marquez, Raul Ruiz De La Hermosa Gonzalez-Carrato, Jesus Maria Pinar Perez, Noor Zaman, 
Mnueer Ahmed, Hakan Johansson, Oscar Gustafsson 


Published by InTech 
Janeza Trdine 9, 51000 Rijeka, Croatia 


Copyright © 2013 InTech 


All chapters are Open Access distributed under the Creative Commons Attribution 3.0 license, which allows users to 
download, copy and build upon published articles even for commercial purposes, as long as the author and publisher 
are properly credited, which ensures maximum dissemination and a wider impact of our publications. After this work 
has been published by InTech, authors have the right to republish it, in whole or part, in any publication of which they 
are the author, and to make other personal use of the work. Any republication, referencing or personal use of the 
work must explicitly identify the original source. 


Notice 


Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those 
of the editors or publisher. No responsibility is accepted for the accuracy of information contained in the published 
chapters. The publisher assumes no responsibility for any damage or injury to persons or property arising out of the 
use of any materials, instructions, methods or ideas contained in the book. 


Publishing Process Manager Ana Pantar 
Technical Editor InTech DTP team 


Cover InTech Design team 


First published January, 2013 


Printed in Croatia 


A free online edition of this book is available at www.intechopen.com 


Additional hard copies can be obtained from orders@intechopen.com 


Digital Filters and Signal Processing, Edited by Fausto Pedro Garcia Marquez and Noor Zaman 
p. cm. 
ISBN 978-953-51-0871-9 


Contents 


Chapter 1 


Chapter 2 


Chapter 3 


Chapter 4 


Chapter 5 


Chapter 6 


Chapter 7 


Chapter 8 


Preface VII 


Maintenance Management Based on Signal Processing 1 
Fausto Pedro Garcia Marquez, Raul Ruiz de la Hermosa González- 
Carrato, Jesus Maria Pinar Perez and Noor Zaman 


Spectral Analysis of Exons in DNA Signals 33 
Noor Zaman, Ahmed Muneer and Fausto Pedro Garcia Marquez 


Deterministic Sampling for Quantification of Modeling 
Uncertainty of Signals 53 
Jan Peter Hessling 


Direct Methods for Frequency Filter Performance Analysis 81 
Alexey Mokeev 


Frequency Transformation for Linear State-Space Systems and 
Its Application to High-Performance Analog/Digital Filters 109 
Shunsuke Koshita, Masahide Abe and Masayuki Kawamata 


A Study on a Filter Bank Structure With Rational Scaling Factors 
and Its Applications 139 
Fumio Itami 


Digital Filter Implementation of Orthogonal Moments 157 
Barmak Honarvar Shakibaei Asli and Raveendran Paramesran 


Two-Rate Based Structures for Computationally Efficient Wide- 
Band FIR Systems 189 
Hakan Johansson and Oscar Gustafsson 


VI Contents 


Chapter9 Analytical Approach for Synthesis of Minimum L2-Sensitivity 
Realizations for State-Space Digital Filters 213 
Shunsuke Yamaki, Masahide Abe and Masayuki Kawamata 


Chapter 10 Particle Swarm Optimization of Highly Selective Digital Filters 
over the Finite-Precision Multiplier Coefficient Space 243 
Seyyed Ali Hashemi and Behrouz Nowrouzian 


Chapter 11 Analytical Design of Two-Dimensional Filters and Applications 
in Biomedical Image Processing 275 
Radu Matei and Daniela Matei 


Preface 


Digital filters, together with signal processing, are being employed in the new technologies 
and information systems, and implemented in different areas and applications. Digital 
filters and signal processing are used with no costs and they can be adapted to different 
cases with great flexibility and reliability. 


This book presents advanced developments in digital filters and signal processing methods 
covering different case studies. They present the main essence of the subject, with the 
principal approaches to the most recent mathematical models that are being employed 
worldwide. 


An approach employing digital filters and signal processing methods based on wavelet 
transforms is presented in order to be applied in the maintenance management of wind 
turbines. It is completed with other techniques as the fast Fourier transform. It leads to a 
reduction of operating costs, availability, reliability, lifetime and maintenance costs. 


The wavelet transforms are also employed as a spectral analysis of exons in 
deoxyribonucleic acid (DNA) signals. These regions are diffused in a noise created by a 
mixture of exon-intron nucleotides. A better identification of exons results in fairly complete 
translation of RNA from DNA. Researchers have proposed several techniques based on 
computational and statistical signal processing concepts but an optimal solution is still 
lacking. The target signal is filtered by wavelet transforms to reduce the noise created by 1/f 
diffused noise. The signal is then processed in a series of computational steps to generate a 
power spectral density estimation graph. Exons are approximated with reference to 
discrimination measure between intron and exons. The PSD’s graph glimpses a clear picture 
of exons boundaries comparable with the standard NCBI range. The results have been 
compared with existing approaches and significance was found in the exons regions 
identification. 


Statistical signal processing traditionally focuses on extraction of information from noisy 
measurements. Typically, parameters or states are estimated by various filtering operations. 
The quality of signal processing operations is assessed by evaluating the statistical 
uncertainty of the result. The processing could for instance simulate, correct, modulate, 
evaluate or control the response of a physical system. A statistical model of the parameters 
describing to which degree the dynamic model is known and accurate will be assumed 
given, instead of being the target of investigation as in system identification. Model 
uncertainty (of parameters) is then propagated to model-ing uncertainty (of the result). 
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Applications include e.g. various mechanical and electrical applications using uncertain 
differential equations, and statistical signal processing. The so-called brute force Monte 
Carlo method is the indisputable reference method to propagate model uncertainty. Its main 
disadvantage is its slow convergence, or requirement of using many samples of the model 
(large ensembles). The use of excitation matrices made it possible to construct universal 
generic ensembles. The efficiency of the minimal simplex (SPX) ensemble is indeed high but 
so is also its third moment. While the standard (STD) maximizes the range of each 
parameter, the binary (BIN) minimizes it by varying all parameters in all samples. The STD 
is the simplest while the SPX is the most efficient ensemble. In the example, the BIN was 
most accurate. For non-parametric models with many parameters, reduction of samples may 
be required. Elimination of singular values (ESV) and correlated sampling (CRS) were two 
such techniques. The presented ensembles are not to be associated to random sampling as a 
method. They are nothing but a few examples of deterministic sampling, likely the best 
ensembles are yet to be discovered. It is indeed challenging but also rewarding to find novel 
deterministic sampling strategies. Once the sampling rules are found, the application is just 
as simple as random sampling, but usually much more efficient. Deterministic sampling is 
one of very few methods capable of non-linear propagation of uncertainty through large 
signal processing models. 


Direct methods for frequency filter performance analysis are considered. The features of the 
suggested performance analysis for signal processing methods are related to consistent 
mathematical models of input signals and the analog and digital filter impulse 
characteristics of a set of continuous/discrete semi-infinite or finite damped oscillatory 
components being used. Simple semi-infinite harmonic and aperiodic signals and 
compound signals, and impulse characteristics of any form can be synthesized on the base 
of components set mentioned. The uniformity of mathematical signal and filter description 
enables one to apply a one-type compact form for their characterization as a set of complex 
amplitudes, complex frequencies and time parameters, and it simplifies significantly 
performance analysis of signal processing by analog or digital filters at any possible input 
signal parameter variation. The signals are directly linked with Laplace transform spectral 
representations, since the damped oscillatory component is the base function of the Laplace 
transform. The application of signal/filter frequency and frequency-time representations, 
based on Laplace transform, allowed developing simple and effective direct methods for 
performance analysis of signal processing of analog and digital filters. The analysis methods 
can be used in substitute of mathematical models as well, where complex amplitudes and/or 
complex frequencies are time functions. 


The frequency transformation for linear state-space systems plays important roles in signal 
processing from both the theoretical and practical point of view. It is applied to high- 
performance analog/digital filters. The frequency transformation easily allows obtaining any 
kind of frequency selective filter from a given prototype low-pass filter, and the frequency 
transformation is also applied to the design of variable filters that enable real-time tuning of 
cut off frequencies and thus have been widely used in many modern applications of signal 
processing. The use of the state-space representation is discussed, which is one of the well- 
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known internal descriptions of linear systems, for analysis of relationships between analog/ 
digital filters and frequency transformation. The state-space representation is a powerful 
tool for synthesis of filter structures with high-performance such as the low sensitivity, low 
roundoff noise, and high dynamic range. The properties to be presented here are closely 
related to the following three elements of linear state-space systems: the controllability 
Gramian, the observability Gramian, and the second-order modes. These three elements are 
known to be very important in synthesis of high-performance filter structures. It is 
developed to the technique of design and synthesis of analog and digital filters with high 
performance structures. It is extended to variable filters with high-performance structures. 


An application in biomedical image processing is done employing an analytical design of 
two-dimensional filters. Various types of 2D filters are approached, both recursive infinite 
impulse response (IIR) and non-recursive finite impulse response (FIR). The design methods 
are done on recursive filters, because they are the most efficient. The proposed design 
methods start from either digital or analog 1D prototypes with a desired characteristic, 
employing analog prototypes, since the design turns out to be simpler and the 2D filters 
result of lower complexity. The prototype transfer function results from one of the common 
approximations (Butterworth, Chebyshev, elliptic) and the shape of the prototype frequency 
response corresponds to the desired characteristic of the final 2D filter. The specific complex 
frequency transformation from the axis to the complex plane will be determined for each 
type of 2D filter separately, starting from the geometrical specification of its shape in the 
frequency plane. The 2D filter transfer function results directly factorized, which is a major 
advantage in its implementation. The proposed design method also applies the bilinear 
transform as an intermediate step in determining the 1D to 2D frequency mapping. In order 
to compensate the distortions of their shape towards the margins of the frequency plane, a 
prewarping is applied, which however will increase the filter order. All the proposed design 
techniques are mainly analytical but also involve numerical optimization, in particular 
rational approximations (e.g. Chebyshev-Padé). Some of the designed 2D filters result with 
complex coefficients. However this should not be a serious shortcoming, since such IIR is 
also used. 


A filter bank structure with rational scaling factors and its applications is presented. The 
frequency patterns of the filter bank is analysed to show how to synthesize scaled signals 
arbitrarily. In addition, possible problems are identified with the structure in image scaling. 
Theoretical conditions for solving the problems are also derived through the input-output 
relation of the filter bank. A design procedure with the conditions is also provided. Through 
simulation results is demonstrated that the quality of scaled images is comparable to those 
of images with typical structures. It is used to potential issues and advantages in utilizing 
the scheme as well as traditional ones in image processing. 


The geometric moments (GMs) are an important aspect of the real-time image processing 
applications. One of the fast methods to generate GMs is from cascaded digital filter 
outputs. However, a concern of this design is that the outputs of the digital filters, which 
operate as accumulators, increase exponentially as the orders of moment increase. New 
formulations of a set of lower digital filter output values, as the order of moments increase, 
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are described. This method enables the usage of the lower digital filter output values for 
higher-order moments. Another approach to reduce the digital filter structure proposed by 
Hatamian, in the computation of geometric moments which leads to faster computation to 
obtain them, is considered. The proposed method is modelled using the 2-D Ztransform. 
The recursive methods are used in Tchebichef moments (TMs) and inverse Tchebichef 
moments (ITMs) computations —recurrence relation regards to the order and with respect to 
the discrete variable. A digital filter structure is proposed for reconstruction based on the 2D 
convolution between the digital filter outputs used in the computation of the TMs and the 
impulse response of the proposed digital filter. A comparison on the performance of the 
proposed algorithms and some of the existing methods for computing TMs and ITMs shows 
that the proposed algorithms are faster. A concern in obtaining the Krawtchouk Moments 
(KMs) from an image is the computational costs. The first approach uses the digital filter 
outputs to form GMs and the KMs are obtained via GMs. The second method uses a direct 
approach to achieve KMs from the digital filter outputs. 


The two-rate based structures for computationally efficient wide-band FIR systems are 
done. Regular wide-band finite-length impulse response systems tend to have a very high 
computational complexity when the bandwidth approaches the whole Nyquist band. It is 
presented in two-rate based structures which can be used to obtain substantially more 
efficient wide-band FIR systems. The two-rate based structure is appropriate for so called 
left-band and right-band systems, which have don’t-care bands at the low-frequency and 
high-frequency regions, respectively. A multi-function system realizations is also 
considered. 


The L2-sensitivity minimization is a technique employed for the synthesis of high-accuracy 
digital filter structures, which achieves quite low-coefficient quantization error. It can be 
employed in order to reduce to undesirable finite-word-length (FWL) effects arise due to the 
coefficient truncation and arithmetic roundoff. It is employed for to the L2-sensitivity 
minimization problem for second-order digital filters. It can be algebraically solved in closed 
form, where the L2-sensitivity minimization problem is also solved analytically for arbitrary 
filter order if second-order modes with the same results. A general expression of the transfer 
function of digital filters is defined with all second-order modes. It is obtained by a 
frequency transformation on a first-order prototype FIR digital filter with the absence of 
limit cycles of the minimum L2-sensitivity realizations, synthesized by selecting an 
appropriate orthogonal matrix. 


The design, realization and discrete particle swarm optimization (PSO) of frequency 
response masking (FRM) IR digital filters is done in detail. FRM IIR digital filters are 
designed by FIR masking digital subfilters together with IIR interpolation digital subfilters. 
The FIR filter design is straightforward and can be performed by using hitherto techniques. 
The IIR digital subfilter design topology consists of a parallel combination of a pair of 
allpass networks so that its magnitude-frequency response matches that of an odd order 
elliptic minimum Q-factor (EMQF) transfer function. This design is realized using the 
bilinear-lossless-discrete-integrator (bilinear-LDI) approach, with multiplier coefficient 
values represented as finite-precision (canonical signed digit) CSD numbers. The FRM 
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digital filters are optimized over the discrete multiplier coefficient space, resulting in FRM 
digital filters which are capable of direct implementation in digital hardware platform 
without any need for further optimization. A new PSO algorithm is developed to tackle 
three different problems. In this PSO algorithm, a set of indexed look-up tables (LUTs) of 
permissible CSD multiplier coefficient values is generated to ensure that in the course of 
optimization, the multiplier coefficient update operations constituent in the underlying PSO 
algorithm lead to values that are guaranteed to conform to the desired CSD wordlength, etc. 
In addition, a general set of constraints is derived in terms of multiplier coefficients to 
guarantee that the IIR bilinear-LDI interpolation digital subfilters automatically remain 
BIBO stable throughout the course of PSO algorithm. Moreover, by introducing barren 
layers, the particles are ensured to automatically remain inside the boundaries of LUTs in 
course of optimization 
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Maintenance Management Based on Signal Processing 
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1. Wind Turbines 


Most of the wind turbines are three-blade units (Figure 1.) [55]. Once the wind drives the 
blades, the energy is transmitted via the main shaft through the gearbox (supported by the 
bearings) to the generator. The generator speed must be as near as possible to the optimal 
for the generation of electricity. At the top of the tower, assembled on a base or foundation, 
the housing or nacelle is mounted and the alignment with the direction of the wind is con- 
trolled by a yaw system. There is also a pitch system in each blade. This mechanism controls 
the wind power and sometimes is employed as an aerodynamic brake. The wind turbine 
features a hydraulic brake to stop itself when it is needed. Finally, there is a meteorological 
unit that provides information about the wind (speed and direction) to the control system. 


1.1. Maintenance in Wind Turbines 


Maintenance is a key tool to ensure the operation of all components of a set. One of the ob- 
jectives is to use available resources efficiently. The classical theory of maintenance was fo- 
cused on the corrective and preventive maintenance [9] but alternatives to corrective and 
preventive maintenance have appeared in recent years. One of them is Condition Based 
Maintenance, which ensures the continuous monitoring and inspection of the wind turbine 
detecting emerging faults and organizing maintenance tasks that anticipate the failure [59]. 
Condition Based Maintenance implies acquisition, processing, analysis and interpretation of 
data and the selection of proper maintenance actions. This is achieved using condition moni- 
toring systems [27, 28]. Thereby, CBM is presented as a useful technique to improve not on- 
ly the maintenance but the safety of the equipments. Byon and Ding [14] or McMillan and 
Ault [50] have demonstrated its successful application in wind turbines, making the CBM 
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one of the most employed strategies in this industry. Another example of the maintenance 
evolution is the Reliability Centred Maintenance. It is defined as a process to determine 
what must be done to ensure that any physical asset works in its operating context [71]. 
Nowadays it is the most common type of maintenance for many industrial fields [25, 26] 
and it involves maintenance system functions or identifying failure modes among others 
maintenance tasks [52]. 


Figure 1. Main parts of a turbine: (1) blades, (2) rotor, (3) gearbox, (4) generator, (5) bearings, (6) yaw system and (7) 
tower [36]. 


1.2. Condition Monitoring applied to Wind Turbines 


Condition Monitoring systems operate from different types of sensors and signal processing 
equipments. They are capable of monitoring components ranging from blades, gearboxes, 
generators to bearings or towers. Monitoring can be processed in real time or in packages of 
time intervals. The procurement of data will be critical to determine the occurrence of a 
problem and determine a solution to apply. Therefore, the success of a Condition Monitor- 
ing system will be supported by the number and type of sensors used and the signal collec- 
tion and processing. 


Any element that performs a rotation is susceptible of being analysed by vibration. In the 
case of the wind turbines, vibration analysis is mainly specialized in the study of gearboxes 
[48, 49] and bearings [81] [85]. Different types of sensors will be required depending on the 
operating frequency: position transducers, velocity sensors, accelerometers or spectral ener- 
gy emitted sensors. 


Acoustic emissions (AE) describe the sound waves produced when a material undergoes 
stress as a result of an external force [35]. They can detect the occurrence of cracks in bear- 
ings [84] and blades [91] in earlier stages. 
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Ultrasonic tests evaluate the structural surface of towers and blades in wind turbines [22] 
[24]. Consistent with some other techniques, it is capable of locating faults safely. 


Oil analysis may determine the occurrence of problems in early stages of deterioration. It is 
usually a clear indicator of the wearing of certain components. The technique is widely used 
in the field of maintenance, being important for gearboxes in wind turbines [47]. 


Thermographic technique is established for monitoring mainly electrical components [72]; al- 
though its use is extended to the search of abnormal temperatures on the surfaces of the blades 
[64]. Using thermography, hot spots can be found due to bad contacts or a system failure. It is 
common the introduction of online monitoring systems based on the infrared spectrum. 


There are techniques that not being so extended, are also used in the maintenance of wind 
turbines. In many cases, their performance is heavily influenced by the costs or their exces- 
sive specialization, making them not always feasible. Some examples are strain measure- 
ments in blades [68]; voltage and current analysis in engines, generators and accumulators 
[67]; shock pulse methods detecting mechanical shocks for bearings [13] or radiographic in- 
spections to observe the structural conditions of the [61]. 


1.3. Signal processing methods 


Fast Fourier Transform (FFT) 


The FFT converts a signal from the time domain to the frequency domain. The use of FFT 
also allows its spectral representation [56]. Each frequency range is framed into a particular 
failure state. It is very useful when periodic patterns are searched [5]. Vibration analysis also 
provides information about a particular reason of the fault origin and/or its severity [43]. 
There is extensive literature demonstrating the development of the method for rolling ele- 
ments. The FFT of a function f(x) is defined as [12]: 


fi foed l 


This integral, which is a function of s, may be written as F(s). Transforming F(s) by the same 
formula, equation (2) where F(s) is the Fourier transform of f(x) is obtained. 


f. f(s)e™ ds (2) 


There are a considerable number of publications regarding the diagnosis of faults for rolling 
machinery that justifies the models and patterns based on the Fast Fourier Transform. Mis- 
alignment is one of the most commonly observed faults in rotating machines, being the sec- 
ond most common malfunction after unbalance. It may be present because of improper 
machine assembly, thermal distortion and asymmetry in the applied load. Misalignment 
causes reaction forces in couplings that are the major cause of machinery vibration. Some 
authors evaluated numerically the effect of coupling misalignment and suggested the occur- 
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rence of strong vibrations at twice the natural frequency [70] [95], although rotating machi- 
nery can excite vibration harmonics from twice to ten harmonics depending on the signal 
pickup locations and directions [53]. 


Faults do not have a unique nature and most of the time, problems ona smaller scale are linked, 
e.g. in the case of misalignment, when an angular misalignment is studied, parallel misalign- 
ment (minor fault) needs to be take into account. Al-Hussain and Redmond reported vibra- 
tions for parallel misalignment at the natural frequency from experimental investigations [4]. 


To facilitate the diagnosis in rolling elements, some companies and researchers tabulate the 
most common failure modes in the frequency domain, so that the analysis can be carried out 
easier. Thus, the appearance of different frequency peaks determines the existence of devel- 
oping problems such as gaps, unbalances or misalignments among other circumstances 
[31].The great advantage of these tables is that the value of the frequency peak is not a par- 
ticular value and may be adapted to any situation where the natural frequency (or the rota- 
tional speed) is known. 


Wavelet transform is a time-frequency technique similar to Short Time Fourier Transform 
although it is more effective when the signal is not stationary. Wavelet transform decom- 
pose an input signal into a set of levels at different frequencies [77]. Wavelet transforms 
have been applied to the fault detection and diagnosis in various wind turbine parts. 


A hidden Markov model is a statistical model in which the system being modelled is as- 
sumed to be a Markov process with hidden states. A hidden Markov model can be consid- 
ered as the simplest dynamic Bayesian network [8]. Ocak and Loparo presented the 
application for the bearing fault detection [57]. 


They are used when a statistical study is required. In these cases, common statistical, i.e. the 
root mean square or peak amplitude; to diagnose faults are employed. Other parameters can 
be maximum or minimum values, means, standard deviations to energy ratios or kurtosis. 
Moreover, trend analysis refers to the collection of information in order to find a trend. 


There are many methods that, as happened with the techniques available for CM, are very 
specific and therefore they are used for very specific situations. Filtering methods, for exam- 
ple, are designed to remove any redundant information, eliminating unnecessary overloads 
in the process. Analysis in time domain will be a way of monitoring wind turbine faults as 
inductive imbalances o turn-to-turn faults. Other methodology, the power cepstrum, de- 
fined as the inverse Fourier Transform of the logarithmic power spectrum [92], reports the 
occurrence of deterioration through the study of the sidebands. Time synchronous averag- 
ing, amplitude demodulation and order analysis are other signal processing methodologies 
used in wind turbines. 


2. Wavelet transform 


The wavelet transform is a method of analysis capable of identifying the local characteristics 
of a signal in the time and frequency domain. It is suitable for large time intervals, where 
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great accuracy is requested at low frequencies and vice versa, e.g. small regions where preci- 
sion details for a deeper processing are required at higher frequencies [23]. The wavelet 
transform can be defined as a signal on a temporal base that is filtered successive times and 
whose average value is zero. These wavelets are irregular and asymmetrical [51]. The trans- 
form has many applications in control process and detection of anomalies. It enables to ana- 
lyse the signal structures that depend on time and scale, being a useful method to 
characterize and identify signals with spectral features, unusual temporary files and other 
properties related to the lack of stationary. When the frequency range corresponding to each 
signal is known, the data can be studied in terms of time, frequency and amplitude. There- 
fore it is possible to see which frequencies are in each time interval, and may even reverse 
the wavelet transform when it is necessary. Previously to the wavelet transform, the FFT 
was able to work with this type of signals in the frequency domain but without great resolu- 
tion in the time domain [38]. 


The wavelet transform of a function f(t) is the decomposition of f(t) in a set of functions and 
Y s: (t), forming a base. It is defined as [88] [66]: 


W,(s,0) =| f(b) .(Bat (3) 


Wavelets transforms are generated from the translation and scale change from a same wave- 
let function w(t), called mother wavelet, which is given by equation (4): 


v=) (4) 


S 


where s is the scale factor, and T is the translational factor. 


The wavelets Y ,, (t) generated from the same mother wavelet function y(t) have different 
scale s and location t, but the same shape. Scale factors are always s>0. The wavelets are di- 
lated when the scale s>1 and contracted when s<1. Thus, the changing of the value s can cov- 
er different ranges of frequencies. Large values for the parameter s correspond to lower 
frequencies ranges or a large scale for y ,, (t). Small values of s correspond to lower frequen- 
cies ranges or very small scales. 


The wavelet transform can be continuous or discrete. The difference between them is that 
the continuous transform provides more detailed information but consuming more compu- 
tation time while the discrete signal is efficient with fewer parameters and less computation 
time [17]. The Discrete Wavelet Transform coefficients are a group of discrete intervals of 
time and scales. These coefficients are used to formalize a set of features that characterize 
different types of signals. Any signal can be divided into low frequency approximations (A) 
and high frequency details (D). The sum of A and D is always equal to the original signal. 
The division is done using filters (Figure 2). 
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Low-pass 


To reduce the computational and mathematical costs due to duplication of data, a sub-sam- 
pling is usually performed, containing the half of the collected information from A and D 
but without losing information. It is common to accompany this information with a graphi- 
cal representation where the original signal is divided in low pass filters and high pass fil- 
ters [15]. When the signals are complex, the decomposition must be to further levels and it is 
not sufficient with two frequency bands. From this need, multilevel filters appear. Multile- 
vel filters repeat the filtering process iteratively with the output signals from the previous 
level. This leads to the so called wavelet decomposition trees (Figure 3.) [2]. By decomposing 
a signal in more frequency bands, additional information is obtained. A suitable branch to 
each signal is highly recommended as more decompositions do not always mean higher 
quality results. 


filter banks 


High-pass 


Figure 2. Decomposition diagram. 


Figure 3. Wavelet decomposition tree. 


The calculation of the Continuous Wavelet Transform starts for an initial time and a scale 
value. The result of multiplying the two signals is integrated into the whole space of time. 
Subsequently, this integral is multiplied by the inverse of the square root scale value, obtain- 
ing a transformed function with a normalized energy. This process is iterative until the end 
of the original signal is reached and must be repeated for all the values of scale that sweep 
the frequency range to be studied. 


Maintenance Management Based on Signal Processing 
http://dx.doi.org/10.5772/52199 


2.1. Wavelet families 


The concept of wavelet has emerged and evolved during the last decades. Though new fam- 
ilies of wavelet transforms are rapidly increasing, there are a number of them that have been 
established with more strength over time. In most situations, the use of a particular family is 
set by the application. 


Daubechies wavelets are the most used wavelets, representing the foundations of wavelets 
signal processing and founding application in Discrete Wavelet Transform. They are defined 
as a family of orthogonal and smooth basis wavelets characterized by a maximum number 
of vanishing moments. The degree of smoothness increases as long as the order is higher. 
Daubechies wavelets lead to more accurate results in comparison to others wavelet types 
and also handle with boundary problems for finite length signals in an easier way [58] [29] 
[60] [94]. Wavelets have not an explicit expression except for order 1, which is the Haar 
wavelet. The inability to present a wavelet equation by a particular formula will be the gen- 
eral trend for almost all types of wavelet families [76]. 


As above mentioned, Haar wavelets are Daubechies wavelets when the order is 1. They are 
the simplest orthonormal wavelets. The main drawback for Haar wavelets is their disconti- 
nuity as a consequence of not solving breaking points problems for its derivates. The Haar 
transform is one of the earliest examples of a wavelet transform and it is supported by a 
function is an odd rectangular pulse pair [33]. Haar functions are widely used for applica- 
tions as image coding, edge extraction and binary logic design and are defined as [46] [41] 
[34] [30]: 


1 
1 O<t<— 
2 


1 
H(t)=4-1 5<t<! (5) 


0 elsewhere 


The main advantages of the Haar wavelet are its accuracy and fast implementation com- 
pared with others methods, its simplicity and small computational costs, and its capacity for 
solving boundaries problems [87]. 


Symlet wavelet transform is an orthogonal wavelet defined by a scaling filter (a low-pass fi- 
nite impulse response filter of length 2N and sum 1). Symlet wavelet transform is sometimes 
called SymletN, where N is the order. Symlet wavelets are near symmetric. Furthermore, 
they have highest number of vanishing moments for a given width [7]. 


Coiflet wavelets are a family of wavelets whose main characteristics are similar to the Sym- 
let ones: a high number of vanishing moments and symmetry. Coiflet family is also com- 
pactly supported, orthogonal and capable to give a good accuracy when the original signal 
has a distortion. The Coiflet wavelets are defined for 5 orders [18]. 
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Biorthogonal wavelets have become very popular because of its versatility, being capable of 
supporting symmetric or antisymmetric signals. They perform very well under certain 
boundaries conditions [97]. Moreover the Biorthogonal wavelet transform is an invertible 
transform. They have two sets of lowpass filters for reconstruction, and highpass filters for 
decomposition [32]. 


Along with the Haar wavelets, the Meyer family is one of the exceptions that can be repre- 
sented by an equation. The Meyer wavelets have numerous applications in the theory of 
functions, solving differential equations, signal processing, etc. [39]. Meyer family has not 
compact support being this one of its drawbacks. It is defined by equation (6) [44]: 


m 
=+0(0o-7) we 
4 ( ) l 


m (a) 
MON 004 = Pi i oe 


Bales 
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(6) 


where 0(w) is a continuously and differentiable function equal to — for w2 — í 


2.2. Wavelet transform applications 


The use of the wavelet transform has been developed over the past two decades focused on 
the process diagnosis and instrumentation. In 1990, Leducq introduces them in the analysis 
of hydraulic noise for a centrifugal pump [45]. Later other authors demonstrates its useful- 
ness for the detection of mechanical failures and the health monitoring control in gears [74] 
[11] [90] [21] [82] [80]. Cracks in rotors [1], structures [73] [63] [89] [10] or composite plates 
[75] has been another exploitation source for wavelet transforms. In 1994, Newland re- 
searches on their properties and applications, and coins the term harmonic wavelet. Har- 
monic wavelets are used for ridge and phase identification in signals [54]. The results 
showed that the cracks found reduced the rotor speed. The effectiveness of wavelets has al- 
so been compared with the envelope detection methodology in the diagnosis of faults in the 
bearings, obtaining results in shorter time analysis [85]. 


Due to its good analytical skills in time regarding the frequency, wavelet transform is a 
guarantee of success in the study of transient processes. Chancey and Flowers [16] managed 
to discover a relation between vibration patterns and the coefficients of a wavelet. Kang and 
Birtwhistle [40] or Subramanian, Badrilal and Henry [78] developed techniques to find prob- 
lems in power transformers. Yacamini [96] proposed a method to detect torsional vibrations 
in engines and generators from the stator currents. 


At present, the development of techniques associated to the scopes mentioned previously 
are still being implemented but others wavelet transforms purposes are emerging, such as 
classification of linear frequency modulation signals for radar emitter recognition [83] or ap- 
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plications to damages caused by corrosion in chemical process installations [86]. As follow 
there is an explanation for some of the most examined in the scientific literature. 


The application of wavelets transforms in wind turbines focuses on the implementation of 
adaptive controllers for wind energy conversion systems. Wavelet transform is capable of 
providing a good and quick approximation. The drivers studied under different noise levels 
achieved higher performances [69]. Other works study the monitoring and diagnosis of 
faults in induced generators with satisfactory results. In these cases a combination of DWTs, 
accompanied by statistical data and energy is proposed. The use of decomposed signals 
spectral components is other highly interesting technique of study. Its harmonic content has 
suitable characteristics to be employed in fault diagnosis as an alternative to conventional 
methods [3]. 


Rolling bearing plays an important role in rotating machines. The choice of a particular 
wavelet family is crucial for the maintenance and fault diagnosis. The location of peaks on 
the vibration spectrum can identify a particular fault. Wavelet decomposition trees are a 
useful tool for this identification. The mean square error extracted from the terminal nodes 
of a tree reports the failure and its size [17]. There are also studies focused on determining 
what type of wavelet is suitable for bearing maintenance [79]. 


The wavelet transform is a good signal analysis method when a variation of time but not of 
space exists. The analysis provides information about the frequency of the signal, being a 
solution for the engine failure detection. There are detection algorithms that identify the 
presence of a fault in working condition and are ahead of the shutdown of the system, re- 
ducing costs and downtimes [19] [20]. These algorithms are independent of the type of en- 
gine used. Other studies in this field, present methods to detect imbalances in the stator 
voltage of a three phase induction motor. The wavelet transform of the stator current is ana- 
lysed. Computationally, these methods are less expensive than other existing and can detect 
faults in an early stage. In the same vein, monitoring fatigue damage has been studied [65]. 


3. Condition Monitoring for engine-generator mechanism 


A novel approach for Condition Monitoring based on wavelet transforms is introduced. A 
system for a mechanism based on an engine and a generator will be shown. It has been de- 
signed to represent any similar mechanism located in a wind turbine, generally in the na- 
celle. These mechanisms are used in cooling devices (generators, gearboxes), electric motors 
for service crane, yaw motors, pitch motors (depending on the configuration) or pumps (oil, 
water) according to the sub systems configurations, ventilators, etc (Figure 4). 


A set of faults are induced in different experiments: ski-slope faults, misalignment faults, an- 
gular misalignment faults, parallel misalignment faults, rotating looseness faults and exter- 
nal noise faults. Pattern recognition is obtained from the extraction of vibration and acoustic 
signals. A Fault Detection and Diagnosis method is developed from the patterns of these 
signals. In order to recognize the patterns, three basic steps have been followed [37]: 
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1. The data acquisition on the testing bench (Figure 5). 
2. The extraction of the features of the experiment using specific algorithms. 
3. A decision-making. 


A classification has been done to obtain the optimal pattern recognitions employing the data 
from Fast Fourier Transform and wavelet transforms applied to the vibrations and sounds 
signals respectively. 


Figure 4. Different locations of a wind turbine where the CM can be used: (1) fans, (2) gear oil pump, (3) oil pump for 
brake and (4) water cooling pump. 


3.1. Case study 


The experiments were made on a mechanism consisting of an engine and a generator linked 
by an elastic coupling joint. The sensors employed were a current sensor, an ambient tem- 
perature sensor, another temperature sensor located in strategic points of the mechanism, a 
vibration sensor; and a sound sensor (microphone). The data obtained by these sensors are 
stored in a data acquisition board, except for the vibration which is collected directly with a 
vibrometer. The software employed was LabView and specific software for vibration pro- 
vided by the manufacturer Kionix. The speed of the engine and its associated frequency 
were set by a frequency variator, and the energy is dispelled using a resistive element. 


The allocation of the vibration measurements were: two points for the engine and two for 
the generator. Points of selection were located at the end of each machine and as close as 
possible to the axis which is the main rotational element of the mechanism (Figure 6). 
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Figure 5. Experimental mechanism. 
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Figure 6. Measuring points. 


The experiments were completed for an average time of 10 seconds each one, and every ex- 
periment was repeated 3 times. Therefore, for each experiment 12 measurements of temper- 
atures, currents, sound, velocities and vibrations were taken (Figure 7). In the case of 


Wot 


vibration, the vibrometer is capable of storing samples for the ‘x’, ‘y’ and ‘z’ axis, in addition 


to a total measurement for the point studied (Figure 8). 


The experiments were carried out in order to identify couplings and misalignments in dif- 
ferent degrees. The engine has 4 rubber clamping (silemblocks), while the generator has 3 
rubbers clamping. The silemblocks were located at the ends, having two on the right side of 
the engine and two on the left side. The generator has them placed in a triangle, two in the 
area closest to the coupling and one at the end. The first experiment recorded under free 
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fault conditions, and the rest of experiments were performed when the silemblocks were re- 
moved from the engine and the generator in order to create the different degrees of decou- 
pling (Figure 9). 


Figure 7. Data collection in LabView. 
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Figure 8. Data collection with Kionix software (vibration). 


The rotational speed is 1500 rpm, i.e. 25 Hz. In order to do an analysis above the natural fre- 
quency, the number of samples was increased from 25 Hz to 125 Hz, being 25 Hz the default 
samples. This guarantees a range 5 times bigger than the natural frequency of the engine. 
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Experiment Type of experiment Data set 
1 Free fault conditions From 1 to 12 
2 isalignment removing silemblocks from the right side of the engine From 13 to 24 


isalignment removing silemblocks from the right side and the front 
3 From 25 to 36 
eft one of the engine 


4 Generation of resistance in the coupling From 37 to 48 
isalignment removing the silemblock from the right side of the 
5 From 49 to 60 
generator 
isalignment removing 2 silemblocks near to the coupling in the 
6 From 61 to 72 
generator 


isalignment removing the silemblock from the right side of the 
7 From 73 to 84 
generator and one from the left side of the engine 


8 Use of a rigid coupling From 85 to 96 


Table 1. Experiments (1500 rpm). 


The FFT of each signal has been developed in Matlab. An algorithm that allows the compari- 
son of two signals for a given frequency was created. The main purpose is to compare pat- 
tern conditions with the signals of the rest of experiments that represent a fault and to 
analyse the peaks found in the natural frequency and its multiples. In some cases it is impor- 
tant to analyse the area located below the natural frequency. Another advantage of the pro- 
gram is that it is possible to obtain the amplitude values for a certain frequency range 
(Figure 10). With a click on a particular peak, the program provides the data. 


Figure 9. Misalignments induced removing silemblocks from the engine and the generator and experimentation with 
a rigid coupling. 
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Values for 25 Hz (natural frequency or 1X), 50 Hz (2X), 75 Hz (3X) and 100 Hz (4X) have 
been taken into account. Frequencies above these values have been discarded. 
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Figure 10. FFT of a vibration signal. 


3.2. Vibration diagnosis and results 


The most common spectrums for engine-generator mechanisms are presented. Examples 
based on the experiments held are shown. 


Ski-slope fault 


A ski-slope fault appears when the spectrum begins at a high level and then it goes down 
slowly (Figure 11). A ski-slope shows a problem with the quality of the sensor. Sometimes it 
happens because the sensor has experienced a transient during the measurement process. 
The transient may be mechanical, thermal or electrical. 


Misalignment faults 


Misalignment fault appears when the centrelines of coupled shafts do not coincide. 

If the misaligned shaft centrelines are parallel but not coincident, then the misalign- 
ment is a parallel misalignment. If the misaligned shafts meet at a point but they 

are not parallel, the misalignment is angular. Most of the cases are a combination 

of them. The diagnosis is based on dominant vibration from the natural frequency 

(1X) at twice the rotational rate (2X), with increased rotational rate levels (3X, 4X, 

etc.) acting in the axial, vertical or horizontal directions. 
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Angular misalignment fault 


Angular misalignment fault produces a bending moment on both shafts and this generates a 
strong vibration at 1X, and some others at 2X and 3X for the axial direction. There will also 
be strong radial components for vertical and horizontal directions (Figure 11). 


Parallel misalignment fault 


Parallel misalignment fault produces a shear force and a bending moment on the coupled 
end of each shaft. High vibration levels at 2X as well as 1X are produced in the radial direc- 
tion. Most often the 2X component is higher than 1X. Depending on the coupling, there can 
be 3X or 4X, even reaching 8X when the misalignment is severe (Figure 11). 


Rotating looseness fault 


Rotating looseness fault will create harmonics or sub-harmonics every 0.5X. Even 1/3 order 
harmonics are possible (Figure 11). 


External noise fault 


It is very common to find a peak in a spectrum that is difficult to analyse. This happens be- 
cause of the vibration from another machine or process. The peak will typically be at a non- 
synchronous frequency (Figure 11). External noise can be verified stopping the machine (or 
varying the speed) and seeing if the vibration is still present or checking local machines for 
the same frequency source. 
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Figure 11. a) Angular misalignment fault (red) and pattern condition (blue), (b) parallel misalignment fault (red) and 
pattern condition (blue), (c) ski-slope fault (blue) and pattern condition (red) and (d) rotating looseness (blue); and 
external noise (red). 
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3.3. Vibration results 


As a rule, the natural frequency (1X) has been kept as the reference. Following the same no- 
menclature, the peaks at 50 Hz, 75 Hz and 100 Hz have been named 2X, 3X and 4X. 
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Figure 12. Vibration for point 1. 
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Figure 13. Vibration for point 2. 


Vibration patterns are different for the four operating points. It has been detected that the 
natural frequency, regardless of its amplitude, tends to predominate in the experiments as- 
sociated with the end points of the set (Figures 12 and 15). Additionally, the generator’s clos- 
est point to the coupling also has a similar pattern (Figure 14). The second point differs from 
the rest, yielding most predominant peaks from the frequency at 50 Hz (Figure 13). To make 
the vibration analysis, it must be taken into consideration not only the appearance of peaks, 
but also the amplitude. The same diagnosis for two experiments can vary its amplitude de- 
pending on the severity of the faults found. The main symptoms appear when peaks at 0.5X, 
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1X, 2X and 3X, sidebands and noise sources are detected. When a failure is studied at an ad- 


vanced stage, peaks at 4X are noticeable (case of rigid coupling). 
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Figure 14. Vibration for point 3. 
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Figure 15. Vibration for point 4. 


The diagnosis of the experiments reveals that the mechanism has a minor looseness which 
causes the appearance of a high peak at the natural frequency in some cases, even under free 
fault conditions. This looseness appears because the engine and the generator are not anch- 
ored directly to the test bench. The assembly was done on a surface that has facilitated the 
removal of the silemblocks when the experiments required it, e.g. to create different degrees 
of misalignment. On the other hand, this action expands the vibration intentionally because 
in this way it is closer to the actual behavior of the nacelle. These frequency peaks change 


their trend in 1X as long as the study advances from the end of the engine to the generator. 
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From point 2, the peak at frequencies as 2X and 3X becomes more significant and sometimes 
exceed the amplitude of the natural frequency. 


The results for experiment 8 are also remarkable. The rigid coupling added causes a severe 
looseness and vibration. The growth of a frequency at 4X and a constant noise over the spec- 
trum is observed. Although it is usual to find sidebands, peaks below 1X and high frequen- 
cy peaks for all this type of experiments, this feature is unique to this last experiment. 
Initially, a similar diagnosis for cases 1, 4 and 8 was expected, but the behavior has been 
slightly different for this reason. 


3.4. Wavelet transform processing approach and results 


Wavelet transforms were employed to analyse the sound signals. As for the Fast Fourier Trans- 
form, an algorithm has been written with Matlab. This program plots and compares two sig- 
nals. Data has been transformed in 5 decompositions named 4 4, d 4, d 3,d and d ,, where each of 
them has an energy rate associated from the original signal (Figure 16). The algorithm also re- 
turns a percentage value per decomposition. These values of energy, the decomposition levels 
attached and the peak amplitudes are examined in order to look for patterns. 


Functions in the time domain can be represented as a linear combination of all frequency 
components present in a signal, where the coefficients are the amount of energy provided by 
each frequency component to the original signal. The main decomposition is associated with 
a , (main or mother wavelet) that usually has the highest energy, though it is not always neces- 
sarily the case. It has a similar pattern to the original signal. The first (d ,), second (d ;), third 
(d ,) and fourth (d ,) transformed signals have decreasing energy rates, being s the original 
signal. Usually a , is the low frequency component of the original signal while d ; is the high 
frequency component, having d , the biggest value. 


It is necessary to verify that the experiments performed at 1500 rpm can be extrapolated to 
other speeds. In the case of wind turbines, most of the engines rotate at speeds close to 3000 
rpm. A certain number of tests were done varying from 500 to 3000 rpm (at intervals of 500 
rpm) in order to ensure the existence of the proportional pattern. 


The results showed that regardless of the speeds or the points of study, all the graphical rep- 
resentations for the different decompositions of energy had the same patterns. Figure 17 in- 
dicates the existence of a similar behavior where only changes the numerical value. The 
biggest ones will correspond to the main signals, while the results for decompositions d ; and 
d , are similar. 


Data can be studied according to the evolution of a single point along the different experi- 
ments or analysing the evolution of the set points for all the experiment. Each row in Figure 
18 contains two graphics, one with the amplitude peaks (left) and the other one with the en- 
ergy distribution of the sound signal (right). The first two graphics correspond to the engine 
end (point 1). The following two graphics are the closest to the coupling (point 2). The third 
row belongs to the points of the generator next to the coupling (point 3), and finally, the last 
two graphics are for the end of the generator (point 4). 
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Figure 16. Wavelet decompositions. 
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Figure 17. Energies at different rotational speeds. 
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Figure 18. Evolution of the frequency peaks and wavelet energy decompositions for each point in experiment 2. 


Based on the distribution of the energy among the 5 different decompositions of every sig- 
nal, the energy distribution for point 1, end of the engine-generator set is ruled by an almost 
similar pattern where each experiment has a maximum of energy in the main signal and a 
minimum for decomposition d ; or d ,. It means that by performing a decomposition of the 
signal, the energy has a closest resemblance to the original value, often exceeding 85% of the 
total energy, remaining a residual percentage for d ; or d } When the experiments are closer 
to the generator (points 2, 3 and 4), the energy is distributed among the 5 decompositions 
and not concentrated in the mother wavelet, as it is for point 1. 


All the decompositions have been registered with their energy maximum and minimum val- 
ues and their patterns distribution. An example for 2 experiments is shown in Table 2. 
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Experiment Main d, d; d, d; Energy 
A 17,19% 9,10% 22,12% 24,95% 26,63% 167,9 
B 82,32% 9,90% 5,17% 1,77% 0,84% 311,8 


Table 2. Energy distribution for experiments A and B. 
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Figure 19. Energy values for point 1. 
Point 2 
400,00 
350,00 
—— Reference 
300,00 
—a— Experiment 2 
8 250,00 »— Experiment 3 
3 
g —x— Experiment 4 
© 200,00 h 
> —*— Experiment 5 
5 150,00 -e— Experiment 6 
—t— Experiment 7 
100,00 —-— Experiment 8 
50,00 
0,00 
1 2 3 4 5 
Decompositions (1=main, 2=D4, 3=D3, 4=D2, 5=D1) 


Figure 20. Energy values for point 2. 


Experiment A is associated to point 2, belonging to the engine and situated close to the cou- 


pling. Experiment B, however, is related to point 1, left end of the assembly. Experiment A 
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has the maximum percentage of energy in d , and the minimum in d , Furthermore, the ex- 
periment B has its maximum in the main signal and the minimum located in d ,. The maxi- 
mum-minimum patterns are d , -d , and main-d , respectively. Numerically, the most 
compensated distribution of energy is close to the coupling (experiment A — point 2) above 
mentioned. The patterns main-d , and main-d , appear for all the cases in point 1. However, 
the same maximum-minimum distribution is smaller for the points 2, 3 and 4. Unlike in 
point 1, there are different patterns for the 8 experiments in these points. Figures 19, 20, 21 
and 22 represent the numerical values of the energy per point and experiment. It must be 
noted that the numerical values are higher or lower, depending on the type of experiment. 
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Figure 21. Energy values for point 3. 
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Figure 22. Energy values for point 4. 
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4. Conclusions 


Wind turbines are complex systems that require a high level of reliability, availability, main- 
tainability and safety. This chapter is focused on to guarantee these correct levels for mecha- 
nisms used in cooling devices for generators and gearboxes, electric motors for service 
crane, yaw motors, pitch motors, pumps, ventilators, etc. 


The mechanism brake of the engine has been simulated linking a generator by a coupling 
joint. The signals collected have been: 


e Vibration. 

° Sound. 

e Current. 

e Temperature. 
e Velocity. 


The experiments have been done in working conditions for different points of the mecha- 
nism and considering the following failures: 


e Misalignment removing silemblocks from the right side of the engine. 


e Misalignment removing silemblocks from the right side and the front left one of the en- 
gine. 


e Induction of resistance in the coupling. 
e Misalignment removing the silemblock from the right side of the generator. 
e Misalignment removing 2 silemblocks near to the coupling in the generator. 


e Misalignment removing the silemblock from the right side of the generator and one from 
the left side of the engine. 


e Using a rigid coupling. 


A fault detection and diagnosis model based on the Fast Fourier Transform applied to the 
vibration signals; together with the wavelet transform applied to sound signals has been de- 
veloped. The model detects and diagnoses correctly 100% of the failures considered. 


It has been observed that for the outer ends of the engine and the generator, the appearance 
of a pronounced peak amplitude at the natural frequency or 2X (vibration) was associated to 
the maximum energy values for the main signal, the most suitable with the original, and 
minimum values for decomposed signals d ; and d , (sound). In contrast, the results obtained 
close to the coupling did not follow a clear trend as the results were conditioned by the type 
of experiment. The numerical values of each peak were also taken into account in the estab- 
lishment of the pattern recognitions, being different for each experiment. The same conclu- 
sion was reached for the energy values. Different models and results were expected because 
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the objective was not to find similar patterns between different experiments, and the tests 
were never performed under identical conditions. The objective was to have different vibra- 
tion patterns and their associated sound models in order to create a catalogue of possible 
scenarios for predictive maintenance in the mechanisms. Thus, it is possible to extend the 
range of possibilities to relate the result of an acoustic signal with the frequency domain us- 
ing the Fast Fourier Transform. 
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1. Introduction 


DNA is found in blood cells carrying nucleus. The DNA is isolated from blood through a 
series of different procedures including heat shock, thermal change and applications of dif- 
ferent chemicals etc. DNA sequence contains chromosomes which further contains genes 
over them. The genes have regions which could translate to protein and the regions which 
don’t perform any contribution in protein production. Both kinds of regions are made-up of 
nucleotides characterized as Adenine, Thymine, Cytosine and Guanine. The order of these 
nucleotides determines the traits, habits and livings of all species. Since with the exponential 
growth of biological data, there is an enormous amount of such data that needs to be trans- 
lated to protein. A successful translation would result in knowing important information 
about species. 


Comparative analysis of computational techniques employed over genetic datasets has given 
very interesting results. We are able to identify species from each other on the behalf of DNA 
properties. A true correct conversion takes to fruitful results. Literature has shown that direct 
comparative analysis is not as useful as approximate estimation. So far, there is no compact sol- 
ution available that could outperform for a robust translation from DNA to RNA. 


It is a common phenomenon that nucleotide sequences in DNA perform a period three 
property [3, 11] due to codon composition and structure in the strand. This fundamental 
characteristic can be exploited to predict the codon regions that help in determination of 
RNA sequences in DNA. This finding is of immense importance as cell growth and function 
is determined by the type of protein the cell produces and helps in drug design and reveal- 
ing genetic disorders as a result of mutation in structure of nucleotide bases (order in which 
they appear over chain). Many approaches have been proposed in literature that addresses 
this open optimization problem in computational biology. 


34 Digital Filters and Signal Processing 


Discrete Fourier Transforms [6, 7, and 8] normally result in spectral leakage that doesn’t pre- 
view the optimal power spectral density estimation. On the other hand, the Short Time 
Fourier Transforms [2, 4] minimize the leakage but are considered useful when we desire to 
have the frequency contents with location information. It can plot the components for time, 
amplitude and frequency of a genetic signal. 


Digital Filters [5, 7, and 13] present the spectral contents of signal around the periodicity 
property of coding regions but don’t specify the frequency time relationship with ampli- 
tude. 


Dosay-Akbulut [14] emphasized the classification of introns in two groups based on RNA 
secondary structure and self splicing ability in variant species using PCR. 


A. Parent et al., [15] describe the importance of coordination between transcription and 
RNA processing that carboxy-terminal domain of RNA polymerase II acts as a common link 
in both. 


Al Wadi et al. [16] used wavelet transforms for forecasting volatility in experimental results. 
M. Hashemi et al. [17] provided Identification of Escherichia coli O157:H7 Isolated from Cat- 
tle Carcasses in Mashhad Abattoir by Multiplex PCR. 


A. Ali et al. [18] have presented a Histopathological Study for development of a model for 
Tumor Lung Cancer Assessing Anti-neoplastic Effect of PMF in Rodents. 


J. Singh et al. [19] proposed a technique for Prediction of in vitro Drug Release Mechanisms 
from Extended Release Matrix Tablets. 


2. Proposed approach 


The proposed approach consists of a series of components that analyze the DNA signal and 
enhances the prediction accuracy of genic regions over DNA sequence. The major steps of 
proposed approach are, 


e Conversion of target DNA stretch to a digital pattern employing an indicator sequence 
e Decomposition of signal using wavelet transforms 

e Calculations of approximate coefficients of signal at level three 

e Calculations of detail coefficients of signal 

e Density estimation of signal 

e Signal analysis for denosing 

e Depiction of original and synthesized signal at level three 

e Histogram estimations of signal 


e Signal extension to a desired length 
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e Shannon entropy calculation of signal 

e Magnitude and power estimation of signal 

e Calculation of discrimination measure for PSD analysis 
° Exon and intron boundaries’ estimation 


As an elaboration, the DNA sequence is passed through a filter that transforms it into a digi- 
tal pattern. This phase is accomplished employing an indicator sequence with the following 
weights for nucleotides, 


Adenine (A) = X (A) = 0.260 
Thymine (T) = X (T) = 0.375 
Guanine (G) = X (G) = 0.125 
Cytosine (C) = X (C) = 0.370 


The corresponding transform becomes 


N 

= —j2akn/N 

Xinaseql] Pa. 2 Xindseq lle (1) 
n=1 


k=1, 2,..,N 


Indicator sequence 


The signal is decomposed employing the wavelet transforms of order three at level three 


y(t) = A(t) + D(t) 
= D cA, (kK) b> (E) + ED (kwal) + X cD (kyw a (É) 
k d k k ! 


= A,(t)+ D,(t) + D; (t) (2) 
= A,(t)+ D,(t) + D,(t) + D, (£) 
= $, cA (K-a, (t) + 2 cD lkw- lt) + DCD (Kwa x(t) + DED, wa lE) 

k k k k 


34 order wavelet decomposition 


The wavelet decomposition passes the signal into a series of low and high pass filters that 
decompose and synthesize the signal for reducing flicker noise (pink noise). 


The signal is then convoluted with a window function (Kaiser Window) defined below, 


1 
nløli-ta-aa¥)* no 0<n<M-1 


w(n) = 


(3) 


0 otherwise 


Kaiser window of length 351 bp 
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Each section of the signal is traversed for calculation of absolute and power values. Each 
segment is plotted over the power spectral graph keeping the period three property main- 
tained at each step. The exon boundaries appear as sharp peaks. The final discrimination 
measure depicts the degree of relevance in exon and introns. 


3. Results and discussions 


A specimen gene pattern S.cerevisiae chromosome IIT (AF099922) has been taken for experi- 
ments over proposed approach. The gene is passed through the series of steps defined, 


At processing stage, the dataset is passed through two kinds of filters. First filter refines the 
data and outputs a data file that purely contains nucleotide characters. Second filter operates 
on output file obtained from first filter application and generates a file that contains numeric 
data. This data is fed into central engine for further processing. 


Figure 1 Shows dataset that contains nucleotide characters and some other characters. This 
is first necessary step because this input when fed into our engine will badly degrade the 
performance and brings false results. 


gttaatgtag cttaataaca aagcaaagca Ctgaaaatgc ttagatg 
jgtaaaat tacacatgcal21 aacctccata gaccggtgta aaatccc 
igacacct tgcctagcca cacccccacg ggactcagca 241 gtgata 
iO01 atttcgtgcc agccaccgcg gtcatacgat taacccaaac taat 
latccaac ttatatgtga aaattcattg 421 ttaggaccta aactc 
(ttagata ccccactatg cttagccata aacctaaata attaaattta 
‘ggcggta60l1 ctttatatcc atctagagga gcctgttcta taatcga 
‘Cagcaaa ccCtaaaaag gtattaaagt /21 aagcaaaaga atcaaa 
tcttata aaagaacatt actataccct ttatgaaact aaaggactaa 
iagtacgc 901 acacaccgcc cgtcaccctc ctcaaattaa attaaa 
‘cgtaaca aggtaagcat actggaaagt gtgcttggaal021 taatca 
Igacactc tgaactaatc ctagccctag ccctacacaa atataattat 
tcgtaca tctaggagct1201 atagaactag taccgcaagg gaaaga 
‘cttttgc ataatgaact aactagaaaa cttctaacta aaagaattac 
ictcgtct1381 atgtggcaaa atagtgagaa gatttttagg tagagg 


Figure 1. Preprocessed dataset 
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Figure 2 represents a data glimpse that contains pure nucleotide characters. 


gttaatgtagcttaat aacaaagcaaagcactgaaaatgcttagatgga 
atgcaaacctccatagaccggtgtaaaatcccttaaacatttgcttaaar 
ggactcagcagtgataaatattaagcaat aaacgaaagtttgactaagt 
attttcggcgtaaaacgtgtcaactataaataaataaatagaattaaaa 
tataatacacgacagctaagacccaaactgggattagataccccactat 
ttaaaactcaaaggacttggcggtactttatatccatctagaggagcct: 
cttcagcaaaccctaaaaaggtatttaagt aagcaaaagaat Caaacat. 
acattactatatcctttatgaaactaaaggactaaggaggatttagtag 
ctcaaattaaattaaacttaacataattaatttctagacatccgtttat: 
ttaatattaaagcatctggcctacacccagaagatttcatgaccaatgg, 
atttatcctactaaaagt att ggagaaagaaattcgtacatctaggagc 
ccttgtaccttttgcataatgaactaactagaaaacttctaactaaaag, 
atgt ggcaaaatagt gagaagatttttaggt agaggtgaaaagcctaac: 
aaCaaaat CaaaaagtaagtttaaattatagcCaaaagagggacagctc 
aaaagcagccaccaacaaagaaagcgttcaagctcaacataaaatttca. 


caacactgttagtatgagt aacaagaattctaattctccaggcatacac. 


Figure 2. Refined dataset 


The EIP indicator sequence transforms the nucleotides in numeric values as per its defini- 


tion. A part of signal is described in Figure 3 below using EIIP indicator sequence as, 


Figure 3. Numeric translation of gene F56F11.5 (AF099922) 


0.1335 0.1335 0.0806 0.1260 0.1260 0.1335 0.1335 0.1340 0.1260 0.1260 0 
0.1340 0.1260 0.1335 0.0806 0.1340 0.1335 0.1335 0.1335 0.1335 0.1335 0 
0.1335 0.1260 0.1260 0.1260 0.1260 0.1260 0.0806 0.1260 0.0806 0.1340 0 
9.1260 0.1260 0.1260 0.1335 0.1335 0.1335 0.1335 0.1335 0.1335 0.1340 0 
0.0806 0.0806 0.0806 0.1260 9.1260 0.1260 0.1335 0.1340 0.1340 0.0806 0 
0.0806 0.1340 0.1335 0.1340 0.1260 0.1260 0.1335 0.1335 9.1335 9.1335 0 
0.1260 0.1260 0.1340 0.1335 0.1335 0.1260 0.0806 0.1335 0.0806 0.1340 0 
0.1335 0.1335 0.0806 0.1340 0.1335 0.1340 0.1340 0.1260 0.1340 0.1340 0 
0.1340 0.1335 0.1260 0.1260 0.1260 0.1335 0.1260 0.1335 0.1335 0.1335 0 
0.0806 0.0806 0.1260 0.1335 0.1335 0.1335 0.1340 0.1260 0.1260 0.1260 0 
0.1260 0.1260 0.1340 0.1260 0.1335 0.0806 0.1260 0.1260 0.0806 0.1335 0 
0.0806 0.1340 0.1335 0.1340 0.1335 0.0806 0.0806 0.1260 0.0806 0.1335 0 
0.1340 0.0806 0.1260 0.1260 0.1260 0.1260 0.1340 0.0806 0.1260 0.1260 0 
0.1340 0.1260 0.1335 0.1335 0.1335 0.1335 0.1335 0.1335 0.1340 0.0806 0 
0.1260 0.1260 0.1260 0.1260 0.1260 0.1260 0.1260 0.1260 0.1340 0.0806 0 
0.1260 0.1260 0.1340 0.0806 0.1340 0.0806 0.1260 0.1260 0.1260 0.1260 0 
0.0806 0.0806 0.1260 0.0806 0.1335 0.1340 0.0806 0.1340 0.1340 0.1260 0 
0.0806 0.1340 0.1260 0.1335 0.1335 0.1335 0.1260 0.1335 0.1335 0.1260 0 


1335 
-1335 
.1260 
.1260 
.1335 
-0806 
-1340 
-0806 
-1340 
-1335 
-1340 
-1335 
.1260 
-1340 
-1340 
-1260 
-1340 
-0806 


The binary indicator sequence is formed by replacing the individual nucleotides with values 


either 0 or 1. 1 stands for presence and 0 for absence of a particular nucleotide in specified 


location in DNA signal, 
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Figure 4 describes the glimpse of binary indicator sequence which is the one of four parts of 
translation of gene file. Only 1's and 0's are visible in this sequence. 


90000007 00000000130001390000000000090090 
900000100100000000000012000100000009090 
a A E S E IES URS A 1 Ee GA Se ce ce 
iE L. a NEG I E S I a A 1 pos Mos RA i es Ba E i A e i e DAES 1 i Naas S iD a es, 1 ES 1E i i N a D i B oe E a a i A ia 
02020 0200 220 2070 Eg i NA i, i NS i LNS i i Nen SA tE i m i A i E i NA i N i es B i ed Daea 1e iNe DO 
900000000000100000100000000013000090090 
a T GE BES 1 IOG i G ck aos ca oe ce a N i N i e, i NAS a NEA a 
O00 00 i 20 8) 4 Oa OG i eoe U i eee GA i eon BA i 0 ee S i g i Ne i gos A 
01000010000001210000000001100000000090 
020702070 02070 70-0 (8 4 O20 i E I i ES BRS i NES i NEY i DES i 8 Oe ae eo 
o0O01T 000000000000000000000000010000090 
0 i S 1R i ca a ee ee ok en ce a Y 1 eg 1 eos D iS Ne i D i 
oo00000000000000000001200000000000909090 
90001 000000000000000000000000000011 
288820 20 242022 i 202028 020 s BS i Ai a i oe 2082078 0 0-8 i eoe 00-8 0 


Figure 4. Binary indicator sequence 


The complex indicator sequence is defined by replacing the nucleotide with 1, -1, iota and - 
iota values. 


Figure 5 shows a portion of gene AF099922 after application of complex indicator sequence. 


0411704117 -17171704117041170-11717170411;04115171717170-1171;04+11;-1;0-11 
237171717171712704117 0411; 0411; 0411; 0411; 0411; 0-11717171;0-11;0411;-1;-1;-1 
117041170411; 0411; -1; 0-11; 0411; 0-117 0-11; -17171;0-11; 0411; 0411;71;-1;041i;- 
1304117 -1;-12-17-17 0-117 041171717170411;71;70411;0411; 0411; 0-11; 041171;7-1;70+ 
171717 -170411; 0-117170-117170-11; -17-1; 0-117 04117 0-1170411; -1;-1717-17041i 
1170411; 0411; 0411; 0411; 0-11; -1;0-117171;-1;0-1170-117171717131717171;0-1i; 
1170-11; -1;0-11;0-1171;0-11;1; 0-11; 0411; 0-11; -1;-1;0-11;1;0411;0411;0411;1 
1370411 7041170-11 70-11 504117 04117 -1 7-1 5151204115 -17041170-117 0-11 7517-17041 
117170411; 041170411; 04+11;0-117171;-171;04+11; 04+11; 0+11; 0+11;0+11;0-11;0-1i; 
237271717171; 0411;7 0411; 170411; 0411; 0411; 04+11;04+11;1;0-11;04+11;-1;04+11;71;-1 
1i;-17041171-7 0-117 0-1171;-17;0-11; -1; -1-1217 0411; 0411; 0411; 041170411; 0+1i; 
137171704117171704+11;7 041170411; 0411; 0411; 0411; 0411; -1; 0-11; 04+11;-1;0411;70+ 
17170411; 041171; -1;0411; 0-11; 0-11; -1;0-117171; 0411; 0411; 0411; 04+11;-1;0-11i; 
13 7041135170-113 5 —1717137151704117 0-117 0411517 -17041170411517130413704+117041i 
11717041171; 0411; 0411; 0411; 04117041171; -1;0-11; 0-11; 0411; 0-11; 0-11704+11;70+ 
23121717 0-11712171217131;: 0411; 0411; -1;04+11;-1; 0411; 0411; 0-11;1;0-11;0411;0 
+11; 0411; 0411; 04+11;0-11;71;0-1171;04+11;04+11;1;04+11;04+11;04+11;04+1171;04+1i;-1 
117170411; 04+11; 0411; 0411; 0-11; 0411; -1;-17171704+1170+11; 0411; 0-11; 0-11;0-1i 
Lis 1515-12-15 -1 70411 704115170-117 04115 -170411717-171717-1204117170-117170 
113;0-11;-1;0-11;-1;-1;-171; 0411; 0411; 0411; 0411; 0411; 0411;-1;1;04+11;0411;0+ 
1370411504117 041170411; -1; 0-117 -1; -170411;70411;-170411;-1;0411;0-1170-11;0 


Figure 5. Complex indicator sequence applied to gene 
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The complex indicator sequence transforms the sequence into four digital patterns with as- 
sociated weights. It is worth mentioning that this indicator sequence provided close range 
estimation for nucleotides in the literature. 


This signal is then passed through the steps of windowed STFT for exonic prediction spec- 
tral analysis. This helps to extend the length of the signal to a target length so that perfect 
analysis could be performed over the signal. 


Figure 6 shows that signal has been extended to a desired length. The length of signal was 
8000 patterns. The convolution method suggests that to perform a better approximation, the 
signal should be extended to 8192 patterns. The signal should be mapped employing Kaiser 
Window of length 351 base pairs. The previous power of two shows a numerical value 4096 
which truncates the signal from its original length. Truncation phenomenon can degrade the 
results and may bring faulty approximation that would lead to differ from the standard 
range of exons. 


Figure 7 depicts the wavelet sketch for db3 wavelet. Scaling and wavelet functions have 
been described. Decomposition of low pass filter and high pass filters have been identified, 
similarly signal synthesis for low and high pass filters have been shown. This sketch demon- 
strates that signal should be passed through these defined filters to further analyze it for 
denoising and enhancement. The upward and downward curves self explains the convolu- 
tion of signal with the window function at desired location of nucleotides. 


== Transformed signal Length L 8000 1 

=== Original signal m 
Next Power of 2 | 8192 
Previous Power of 2 | 4096 
Desired Length | 8192 


Figure 6. Signal extension to desired length 


39 


40 Digital Filters and Signal Processing 


db Wavelet --> db3 


Scaling function phi Wavelet function psi 
4 4 
0.5 4 
0 
it) 1 2 3 4 5 
Decomposition low-pass filter Decomposition high-pass filter 
0.8 
0.6 
0.4 
0.2 
02 $ $ | 
-0.4 
0 14 2 3 4 $ 0 1 3 4 $ 
Reconstruction low-pass filter Reconstruction high-pass filter 
08 
06 4 
O. 4 
0.2 
0 
-0.2 
-0.4 a -L 4 =L 4 
0 1 3 4 5 


Figure 7. Wavelet of db3 sketch 


Figure 8 shows a wavelet tree for Shannon entropy of signal. There is a tree structure for the 
nodes depicting different position factors. Colored coefficients for terminal nodes can be ob- 
served. The first rectangle at the right top shows the analyzed signal at different nucleotides 
places (diffusion of bases at DNA strand). Calculation of Shannon entropy would assist in 
further identification of boundary values for individual nucleotides at power spectral densi- 
ty estimation graphs. 


The digital signal passes through refinement stages. First, the sequence was obtained as a 
raw data which was purified to access only nucleotides bases without degrading factor. This 
is termed as an important process because any kind of unwanted characters may lead to dif- 
ferent set of nucleotides values that would be away from actual results. 


The digital signal under discussions contains 8000 base pairs. The same dataset was used ex- 
tensively in literature by other researchers and it is being used as a bench mark. The spectral 
estimation graph reveals that it contains five exonic regions at different nucleotides ranges. 
Identification of these ranges close to standard range demands to denoise the signal and se- 
lection of an appropriate window function that could be used for perfect convolution. The 
standard convolution requires to multiply the signal with a portion of window function, this 
is the reason that signal was extended to a power of two to make it to desired length. Each 
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frame of the signal is calculated numerically equal sized so that power spectral graph is uni- 
form in all characteristics. 


For discrete wavelet transforms of order three, the signal is decomposed and synthesized. 
These db3 performs the quick vanishing of coefficients for approximate and detail patterns. 


Figure 9 shows a glimpse of original signal. There are 8000 base pairs shown in the form of a 
digital pattern. Cumulative histogram of signal shows different range of weight values as- 
signed to nucleotides base pairs. It can be seen that nucleotides with numerals higher than 
0.25 have high frequency while those between 0.1 and less than 0.25 have lower frequency. 
The individual histogram also shows three separate characterizations of nucleotide weight 
values. The standard deviation has been found to be 0.09037, median of absolute deviation 
is 0.11 while mean absolute deviation is 0.07843. The maximum range is 0.375 while mini- 
mum range is 0.125 and the average range is depicted as 0.25. 


Wavelet Tree Analyzed Signal : length = 8000 
a i amaai IMs 


0.35 


03 


0.25 


0.2 


015 


1000 2000 3000 4000 5000 6000 7000 8000 


Colored Coefficients tor Terminal Nodes 


Node Action Result 


frequency ordered coefficients 


Scale of Colors from Min to Max 


Figure 8. Wavelet tree for Shannon entropy 
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b (8000 values) analyzed at level 3 with db3. Components: 1 --> 8000 


Original signal 
0.35 
03 
0.25 
0.2 
0415 
1000 2000 3000 4000 5000 6000 7000 8000 
Histogram Cumulative histogram 


04 08 

03 06 

02 o4 

04 0.2 

0 + z : 5 7 

0415 0.2 0.25 03 0.35 

Mean f 0.2941 | Maximum 0.375 | Standard deviation 009037 | 
Median | 0.26 Minimum f 0125 | Median absolute deviation F 011 ] 
Mode | oaas ë O Range _ =o | Mean absolute deviation [ 0.07843 ] 


Figure 9. Original signal at level 3 


The histogram calculates the frequency of nucleotides bases in the signal. Since the signal 
was mapped with an enhanced indicator sequence which assigns perfect weights to nucleo- 
tides bases, histogram of such a signal is uniform. It is observed that almost half of the signal 
is diffused in first band and half in the second band. First half band shows the smaller histo- 
gram values (frequency components) while second half band depicts some larger histogram 
values. 


The individual histogram components dependant over the individual nucleotide bases, for 
instance, the numeric value of Adenine is 0.260, which is plotted against the other numeric 
values for bases in individual histogram. Depending over the weights assigned to Thymine 
and cytosine, the histogram shape may change. 


It is also important to note that histogram of frequency components present the redundancy 
of bases in the digital pattern. This repetition depends over the order of nucleotides in DNA 
sequence which defines the habits, traits and other characteristics of species. 


Figure 10 shows the synthesized signal at level three. Like the original signal, the synthe- 
sized signal owns the same histogram characteristics. There are 8000 base pairs shown in the 
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form of a digital pattern. Cumulative histogram of signal shows different range of weight 
values assigned to nucleotides base pairs. It can be seen that nucleotides with numerals 
higher than 0.25 have high frequency while those between 0.1 and less than 0.25 have lower 
frequency. The individual histogram also shows three separate characterizations of nucleo- 
tide weight values. The standard deviation has been found to be 0.09037, median of absolute 
deviation is 0.11 while mean absolute deviation is 0.07843. The maximum range is 0.375 
while minimum range is 0.125 and the average range is depicted as 0.25. 


b (8000 values) analyzed at level 3 with db3. Components: 1 --> 8000 


Synthesized signal 


0.35 


03 


a 


Histogram Cumulative histogram 


7000 8000 


Str deviaton oases? 
Median Minimum Median absolute deviation 011 
Mode Range Mean absolute deviation 0.07843 


Figure 10. Synthesized signal at level 3 
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The synthesized signal shows the same histogram even after its decomposition. The synthe- 
sized signal is perfectly reconstructed by employing discrete wavelet transforms. The ap- 
proximate and detail coefficients of signal are obtained in passing through a series of filters. 
These digital filters have been defined and constructed using Matlab. The decomposed sig- 
nal is addition of approximate and detail coefficients at level three along with detail coeffi- 
cients at level two and level one. 


As for as, we decompose the signal, the components are loosely packed. 


Figure 11 depicts the signal decomposition into approximate and detail coefficients. Symbol 
s represents the original signal. Approximate and detail coefficient at level three show the 
reduced complexity in the signal. 


Decomposition at level 3: s = a3 + d3 + d2 + d1 . 


0.35 i i 
03 | 

s 025 jj 
HATA HIATT 
015 | | | | | | | | | | i 
04 T T T T T T T 
o3 h 
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d 


1000 2000 3000 4000 5000 6000 7000 8000 


Figure 11. Signal decomposition 
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Figure 12 presents the histogram for approximate and detail coefficients. At level one, the 
concentration of components is less than other levels. Level two shows that signal compo- 
nents are more concentrated. At level three, the signal components are more closely packed. 
Likewise, the histogram for approximate coefficients presents the same phenomenon. At 
level one, the concentration of components is less than other levels. Level two shows that 
signal components are more concentrated. At level three, the signal components are more 
closely packed. It can be observed that original signal and synthesized signal contain the 
same number of components. The concentrations of signal components are uniform over 
these histograms’ plots, which depict the perfect reconstruction of digital signal. 


b (8000 values) analyzed at level 3 with db3. Components: 1 --> 8000 


Figure 12. Histogram of signal 
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Figure 13 shows the density estimation of approximate and detail coefficients. The density 
estimate of original signal shows the numerals for nucleotides present in the signal in digital 
format as a general. The approximate coefficients at level three presents a sharp peak at 
some 0.25 points. The signal remains uniform through the course except at another peak val- 
ue ranging from 0.37 to the end of the signal. The density estimation for detail coefficients at 
level one shows the same sharp peak around 0.27 points. The same peak can be observed 
around 0.40 at level two. At level three, the phenomenon is same but the signal components 
are loosely packed than level two. At granular level, the components are more packed at lev- 
el one than other levels. 


Data Binned Data 
0.35 
ost 2000 
0.25 
ool 1000 
045 
0 
045 0.2 0.25 03 0.35 
64 bins - Histogram Density Estimate 
200 
2000 + 
100 
1000 
o 0 
015 0.2 0.25 0.3 0.35 015 0.2 0.25 0.3 0.35 
Approximation 


Details coefficients Details 


Figure 13. Density estimation of signal 
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Figure 14 shows the resultant denoised signal. It is obvious that preview of detail coeffi- 
cients at level three shows the loosely packed signal components. The original signal is rep- 
resented in red color. The threshold coefficients are shown in vertical bars for all nucleotide 
range (8000 base pairs). The coefficients at detail level depicts a hierarchy of packed, loosely 
packed and more loosely packed components, which shows a gradual improvement in the 


signal for denosing. 


Original details coefficients 


1000 2000 3000 4000 5000 6000 7000 8000 


Figure 14. Denoised signal 
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Figure 15 shows the approximate coefficients at level three. A sharp gradual change can be 
observed in a commutative histogram. The peaks are more pronounced at from point one to 
onwards. In another histogram, the peaks are not much visible around first 0.6 points, there 
is a sharp gradual increment in the bars reaching the maximum of 0.07 points then a gradual 
decrement is observed leading it to point one. The peaks are less pronounced after this 
point. The coefficients of approximation at level three show the signal as loosely packed 
components. 


It can be observed that detail coefficient at level one are packed showing more concentration 
of nucleotides while detail coefficients at level two are loosely packed. The coefficients at 
level three are more significant than other levels, which represents that the signal is filtered 
for refinement. The signal was passed through a series of filters for the wavelet db3 which 
denoised the signal as a result of reconstruction of signal. 


b (8000 values) analyzed at level 3 with db3. Components: 1 --> 8000 


Coefficients of approximation at level 3 
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Figure 15. Approximation coefficients at level 3 
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Figure 16 shows the detail coefficients at level three. A sharp gradual change can be ob- 
served in a commutative histogram. The peaks are more pronounced at from point one to 
onwards. In another histogram, the peaks are not much visible around first 0.6 points, there 
is a sharp gradual increment in the bars reaching the maximum of 0.07 points then a gradual 
decrement is observed leading it to point one. The peaks are less pronounced after this 
point. The coefficients of detail at level three show the signal as loosely packed components. 


b (8000 values) analyzed at level 3 with db3. Components: 1 --> 8000 


Coefficients of detail at level 3 
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Figure 16. Coefficients of detail at level 3 


It can be observed that detail coefficient at level one are packed showing more concentration 
of nucleotides while detail coefficients at level two are loosely packed. The coefficients at 
level three are more significant than other levels, which represents that the signal is filtered 
for refinement. The signal was passed through a series of filters for the wavelet db3 which 
denoised the signal as a result of reconstruction of signal. 


Table 1 presents the nucleotide range for exons. Clear differences can be observed as a com- 
parative analysis of various approaches. Binary and EIIP methods show a wide range differ- 
ence compared with the standard range. Complex method results are better than the first 
two approaches. Digital filters behave accordingly. The proposed approach has more signifi- 
cant results than other prevailing approaches. 
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Method E, E, E; E, E; 
Binary Method 656-1206 2406-3106 3806-4406 5306-5806 7106-7706 
EIIP Method 706-1206 2206-2906 3906-4406 5206-5806 7206-7706 
Complex Method 750-1100 2600-2906 3600-4406 5206-5706 7106-7600 
Filter 1 (Anti-notch) 656-1206 2450-3106 3806-4450 5306-5850 7106-7750 
Filter 2 (Multistage) 706-1250 2206-2950 3906-4450 5206-5850 7206-7706 
proposed Method 750-1050 2450-2906 3950-4380 5206-5600 7220-7680 
CBI Range 928-1039 2528-2857 4114-4377 5465-5644 7255-7605 


Table 1. Range of exons for different methods 


4. Conclusion 


Bioinformatics is a very rapidly emerging field of research. The genome sequence analysis is 
an interesting and challenging task that needs great attention. The analysis brings very 
promising relevance between species. The proposed approach provides a way to better 
identify the genetic regions in mixture of exon-intron noise. The focus directed to minimize 
the leakage of frequency contents by adoption of an optimal indicator sequence. We also re- 
duced the signal noise by using Kaiser Window function with length 351 base pairs. The 
spectral density estimation was enhanced with application of wavelet transforms. The pro- 
posed dimensions reduced the noise and increased the sharp peaks of exons in density 
graphs. We have observed significant improvement in results as a comparative analysis be- 


tween existing techniques and compared the results with strands NCBI range. 
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1. Introduction 


Statistical signal processing [1] traditionally focuses on extraction of information from noisy 
measurements. Typically, parameters or states are estimated by various filtering operations. 
Here, the quality of signal processing operations will be assessed by evaluating the statistical 
uncertainty of the result [2]. The processing could for instance simulate, correct, modulate, 
evaluate, or control the response of a physical system. Depending on the addressed task and 
the system, this can often be formulated in terms of a differential or difference signal processing 
model equation in time, with uncertain parameters and driven by an exciting input signal 
corrupted by noise. The quantity of primary interest may not be the output signal but can be 
extracted from it. If this uncertain dynamic model is linear-in-response it can be translated into 
a linear digital filter for highly efficient and standardized evaluation [3]. A statistical model of 
the parameters describing to which degree the dynamic model is known and accurate will be 
assumed given, instead of being the target of investigation as in system identification [4]. Model 
uncertainty (of parameters) is then propagated to model-ing uncertainty (of the result). The two 
are to be clearly distinguished — the former relate to the input while the latter relate to the 
output of the model. 


Quantification of uncertainty of complex computations is an emerging topic, driven by the 
general need for quality assessment and rapid development of modern computers. Applica- 
tions include e.g. various mechanical and electrical applications [5-7] using uncertain differ- 
ential equations, and statistical signal processing. The so-called brute force Monte Carlo 
method [8-9] is the indisputable reference method to propagate model uncertainty. Its main 
disadvantage is its slow convergence, or requirement of using many samples of the model 
(large ensembles). Thus, it cannot be used for demanding complex models. The ensemble size 
is a key aspect which motivates deterministic sampling. Small ensembles are found by 
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substituting the random generator with a customized deterministic sampling rule. Since any 
computerized random generator produces a pseudo-random rather than a truly random 
sequence, this is equivalent of modifying the random generator to be accurate for small 
ensembles of definite size, rather than being asymptotically exact (infinite ensembles). Correct- 
ness of very large ensembles is of theoretical but hardly practical interest for complex models, 
if the convergence to the asymptotic result is very slow. 


2. Modeling uncertainty of signals 


2.1. Problem definition 


Suppose the (output) signal y(x, t) € RK of interest is generated from the (input) signal x(t) € RK 
passing through a dynamic system H, with parameters a, € K, b, € K, 


[$an y- 30.0" a =1. (1) 


The model is given in n=u +v +1 uncertain parameters, which can be arranged in a column 
vector q=(bo =o b, a = a)". For systems continuous-in-time (CT), D=0, is the differen- 
tial operator in time while for systems discrete-in-time (DT), D=A™' is the negative unit 
displacement operator, A ~'x,=x,_;. There are several approximate methods to sample CT 


systems to DT systems, see [3] and references therein. The discretization techniques are beyond 
the scope of this presentation and DT systems will be assumed. If u 21, there is feedback in the 
system which results in an impulse response h (q, t) of infinite duration. For finite accuracy 
however, the duration is finite. The system is linear-in-response, 
y(x=ax, + Bx, t)=ay(x,, t)+By(xX», t). Most importantly, the system is non-linear-in-param- 
eters if u21. This is the typical situation addressed here. 


Systems of the form in Eq. 1 may be directly realized as digital filters, y(q, x, t)=h (q) * x(t), 
where * denotes the filtering operation. The coefficients b, and a, are the numerator and 
denominator coefficients of the filter with impulse response h (q), respectively. Its z-transform 
H(q, z) is obtained with the substitution A — z. The parameterization can be changed to for 
instance gainK, poles p, and zeros z,, or poles p, and residues r,, 


v 


| Dat TM le-a)/(t-2) 
¥(2)=H(2)X(2): nS Hern) ren) O 
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The parameterization should be carefully chosen as it affects the convergence rate of Taylor 
expansions (section 3.1) as well as the physical interpretation. The parameters and their 
statistics are preferably extracted from measurements using system identification techniques 
[4]. Note that complex-valued poles and zeros are conjugated in pairs [10]. 


The problem to be addressed is the statistical evaluation of any function 
g(y(t)=h (q, t) * x(t), given statistical models of q and x. It will here consist of evaluating its 


time-dependent mean (g(y)) and standard deviation ¥([¢(y)—-(¢(y))/*). Without loss of 
generality, the analysis will be made for g(y)=y. Digital filtering will be utilized for evaluating 
samples of the model, i.e. filtering with definite sets of q and signals x. 


2.2. Nomenclature 


Statistical expectations of any signal, model or function g(q) over finite discrete E as well as 
continuous ensembles or probability distributions (no subscript) are defined as, 


Ge = 


(s) = Js(a)f,(a)aa. (3) 


A 
Samples of q are labeled q, with their components organized in columns. Sample indices will 
N 
be given as superscripts in parenthesis, eg. q“ is a column vector denoting the k-th sample 
of parameter q. Variations from the mean are written as 6q(7)=q-(4)£): 


Only uniform (UNI) and normal distributions (NRM) will be utilized. Either the mean and 
standard deviation, or the interval in brackets will be given in parenthesis, e.g. 
q~UNI(0.5,1/ 2/3)=UNI((0, 1]). Statistical moments M,=4((6q,)*) carry the information 
contained in the marginalized probability density functions (pdf) 
f (6q;)= | f (Oq)dq, dq;_44q;,, dq, where Q denotes the sample space. While M ® describes 


Q 
the width of f;(5q;), M;® is related but different to its skewness [11]. Further, the shape is 


reflected in M A similarly to the curtosis [11]. Since UNI(0,1) and NRM(0,1) are normalized 
and symmetric f ,(5q;)= f;(—6q;), M 4, and M =, Their differences are first reflected in 
their fourth moment, M{=1/4y5, 1/2¥3=(0.11, 0.29) for UNI(0,1) and NRM(0,1), respec- 
tively. The maximum variation of the parameter q; is expressed by the range 


M;=lim|M, | =max(|5q;|). Dependencies are expressed in mixed moments 


paa 
(ôq) (6 in)? )(g) The discussion will be limited to correlations described by the covariance 


matrix cov(q)=(5q6q '), where the vector multiplication is an outer product. 
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Matrix size will be indicated with subscripts, e.g. V „xm is a matrix of n rows and m columns 


nxm 
with elements V its j=l, ...n and k=1, ...m. The identity matrix will be denotedI, while 
matrices with equal elements (i) will have their size attached, (i,,.,,) jk =i. For a matrix (vector) 
D, diag(D) is a vector (diagonal matrix) with components (diagonal elements) equal to the 
diagonal elements (components) of D. The trace of a matrix is denoted Tr. 


A method will be stated intrusive if manipulations of the model are required. For the targeted 
highly complex models, it will be assumed that the computational cost for their evaluation 
dominates all other calculations. The efficiency p of any method will accordingly be defined 
by the least required number of evaluations of the original model. 


2.3. Fundamentals of non-linear propagation of uncertainty 


Linearity in parameters (LP) is to be distinguished from linearity in response (LR), 


LR: yļ(q ax + ayX5,t) = ayy (q,x,,t) + ay (q, x,t), Wi ite 


, (4) 
LP: y(qy+qyxt) = y(qyxt)+C(xt) (49-1) Vora 


for some vector C Different concepts of linearity are used, 


nxl* 
y(q, +4, x, t)#y(qy, x, t)+ y(q, x, t) for LP models. Strictly speaking, LP denotes models that 
are affine, i.e. written as linear combinations of their parameters. Most constructed systems 
are designed to be as close to LR as possible while most models are not LP. There is hence no 
contradiction in non-linear (LP) propagation of uncertainty with linear (LR) digital filters, as 
here. 


For non-linear propagation of uncertainty, the asymmetry of the resulting pdf is central. It can 
be expressed as a lack of commutation of non-linear propagation and statistical evaluation of 
a center value (- ç ), as measured with the scent [12], 


£=yc(9)-y(4c)- (5) 


The method for evaluating the center is left unspecified, as there are several alternatives. The 
most common choice is to use the mean, -ç =(- ). The lowest order approximation of the scent 
can then be obtained by calculating the expectation of a Taylor expansion (section 3.1), 
C=Trlcov(q) - H(y)]/2, where H(y) ik =07 y / dq q; is the Hessian matrix signal of y, evaluated 


at (q). The scent is related to the skewness y =(5y*)/(5y?)>”. The additional asymmetry caused 
by the non-linearity of the model is measured with the scent but differently. The scent 
addresses how parametric uncertainties are propagated and not how the result is distributed, 
e.g. C=0 for all LP models for which y may attain any value. A finite scent thus implies the 
model is not LP, but not the reverse. The scent should not be confused with bias. Bias is a 
property of an estimator, while scent is a property of a model. For every model, such as the 
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REF (section 6.1), many different estimators of y-(q) can be used, e.g. the different ensembles 
in section 5.6, see result in Fig. 5 (left). Consequently, an unbiased estimator of y-(q) correctly 


accounts for rather than ignores its finite scent, or deviation from y(gc). 


The scent is important since yç and not y(q,) is the main result utilized in applications. The 
corresponding difference [13] in the standard deviation M A from its linearized approximation 
VV y? cov(q)V y, with (V y) =ð; y(t,), affects the confidence in the result. Its accuracy is 
usually less critical. An accurate evaluation of the scent is perhaps the strongest feature of the 


unscented Kalman filter, which provides the foundation for the presented approach as well as 
the origin of the term ‘scent’. 


3. Conventional methods 


A brief resume of the most traditional related methods of uncertainty propagation, applicable 
to signal processing models, is here given together with their pros and cons. Advanced 
intrusive methods like e.g. polynomial chaos expansions [14-15] not directly related to the 
proposed method are omitted. 


3.1. Taylor expansions 


The indisputable default methods of uncertainty propagation are based on Taylor expansions. 
These methods are intrusive if the differentiations are made analytically. Convergent series 
require regular differentiable models and numerical or analytical complexity make them error 
prone. Their applicability is therefore limited for complex models. 


The transfer function H (q, z) of the digital filter can be expanded in a Taylor series, 


12 4 k 12 3H 
5H (q,z) =H(q,z)-H((q),z) > —(67'V,) H(q,z) e > Pao 


om +... P 
= oq E") ((q),2)+Tr4 [ 599” BP) (a),2)} +... 


This defines n sensitivity systems (column vector) E Pq), z), n(n+1)/2 unique quadratic 


variation systems (matrix) E ay, z), and so on. These variation systems differ (intrusive) 
from H (q, z) but may nevertheless be realized as digital filters [3,7,10], just as H (q, z). The 
corresponding variation of y(q, x, t)=h (q, t) * x(t) is given by, 


5y(q,x,t) = ar [eO ((a)et) x(t) J+ T6709" ]-]e)((g),t)*x(4) | + (7) 
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where e (gy, t) are the impulse responses of the systems E E) ((q), z). Utilizing digital filters 
with impulse responses ea), t), the differentiations are conveniently done once, and not 
repeatedly for every signal x(t). The linearity in parameters of the model can easily be studied 
for many different input signals x(t), by evaluating e Kgy, t) * x(t). Due to the large number 
of variation systems, higher order perturbation analyses rapidly become intractable though. 
The established method is limited to linearization (LIN) [16] (e®). Tt will always incorrectly 
yield vanishing scent, C=0. A first order estimate of Ç is instead given by the expectation of 
second term in Eq. 7, C=Tr[cov(q)H((q), t)1/2, where the matrix of Hessian signals 
H((qy, t)=e ((q), t) * x(t) is obtained with repeated digital filtering. 


3.2. Brute force Monte Carlo 


Monte Carlo (MC) methods [8-9], or random sampling of uncertain models was originally 
introduced and phrased ‘statistical sampling’ by Enrico Fermi already in the 1930's [17]. The 
MC methods realize uncertain signal processing models in finite ensembles. Every ensemble 
consists of a possible set of well-defined model systems, all (usually) having the same structure 
but slightly different parameter values. In the original so-called brute force Monte Carlo 
method, each set of parameters is assigned to the output of random generators with appro- 
priate statistics. The convergence to the assigned statistics is very slow [5] but it is asymptot- 
ically exact and the required number of samples is essentially independent of the number of 
parameters. Hence it does not suffer from the curse-of-dimensionality of many other methods. 
The outstanding simplicity in application is likely the cause of its popularity, just as the slow 
convergence or low efficiency is the main reason for its failures. 

In MC, arbitrary distributions and dependencies are usually obtained by means of transfor- 
mations of samples of elementary distributions. Independent samples qe of any probability 
density function (pdf) ọ(x) can be constructed with the inverse transform method [9]. It 
consists of a calculation of the inverse of its cumulative distribution function (cdf) ®(y) and 


A 
generation of a uniformly distributed random sequence z™”, 


y 
PO =07(209), (y)= f @(x)dx, z~UNI(0,1), k=1,2,...m. (8) 
Covariance may be included with an appropriate transformation of samples of canonical 
parameters g: q=U 'Sq with cov(q)=1, 


uu =UU! =! 


cov(q) = (696q" ) = (usoq(ur’sa) ) = US (sq6q" )SU as", i. Wik (9) 


The matrices S, U are found by calculating the eigenvalues (S ?) and eigenvectors (U) [11] of 
cov(q). This transformation makes the marginal pdfs f,(q,) to differ substantially from the 


univariate pdfs o, of the independent but scaled parameters Sġ,, 
P k P P Ik 
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Flak) = fø ([u] 2 ([Ua], ):--#([Ua],)---¢, ([Ua], Jaq, dg, 444,..1°°°G, # Ala) ifU I. (10) 


All ġ, are hence mixed according to U. Dependencies are thus difficult to account for. One 


rare exception is provided by the multinomial distribution [9]. It is often better to assign the 
pdfs to the canonical parameters in the original instead of the canonical basis. The transfor- 


mation then reads q: q=U TSU q. As required, it leaves cov(q) invariant. The marginalization 
in Eq. 10 changes accordingly, U > SU TSTU. Since the transformation U "SUS ™ of Sq, 
contains cancelling operations U, U” and S, S7, it is generally less distorting than U 7. 
Indeed, if the commutator [S, U7 ]=SU7-U'S vanishes, U'SUS “=I. The transformation 


UT must satisfy the stronger criterion U =I to avoid mixing. For any transformation q > Wq, 
an indicator of mixing of the components of q is given by, 


7 _1¢ max|W,.|~min|W,, OR 3 
(w) = 1 w] [0,1], [w= Dwal- (11) 


A simple example illustrates that the mixing effect can be considerable, even for minute 
correlations. Assume a model has two parameters with a covariance matrix, 


0.90 0.10 1(1 1) 5 (1 0) TaSi) = UNI([0,1)) 
cov(a)=(9%p boo) eB Ae 1 sal h (Sù) = uNI([0,v08 |) a2) 


Large rotations are required because the canonical variances S f are similar, i.e. cov(q) is almost 


degenerate. As shown in Fig. 1, the large rotations mix the assigned pdfs #,(Sq,) to marginal 


pdfs f;,(q,.) beyond recognition for the transformation U ' but not for U TSU S7. 


Figure 1. Left: The sample space of independent scaled parameters (/ :q,=5q,) (Eq. 12), and of the two transforma- 
tions (u 7) (rotated) and (uTsus~) (skewed and tilted). Right: Assigned pdfs $,(S4,) (dashed) and obtained margin- 
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al pdfs f (qy) (solid) ) with mixing ¥(U")=1.00 and ¥(U "SUS~')=0.058, and magnified upper transition region 
(inset). 


Specifying both marginal probability distributions and covariance is either redundant or 
inconsistent, as the latter is uniquely determined by the former. Nevertheless, this reflects the 
typical available information for signal processing applications. The moments can be accu- 
rately determined [4] for sufficiently large data sets but the joint distribution f (q) is hardly 
ever known with any precision. Some of its properties are usually assigned, with varying 
degree of confidence. For instance, the allowed maximal range M () of the parameters of 
digital filters is given by stability constraints. The transformation technique above is well 
adapted to these facts, since the covariance is prioritised. The transformation q=U "SU q will 
be utilized in section 5.2 to include correlations with limited mixing of the statistics assigned 


to independent normalized canonical parameters q. 


3.3. Refinements of Monte Carlo 


To increase the efficiency of MC, the original brute force sampling technique has been further 
developed in mainly two directions: model simplification and sample distribution improve- 
ment. In response surface methodology (RSM) [18], the model is replaced by a simple approx- 
imate surrogate model. A model of order v may be found by applying linear (with respect to 


: ‘ ‘ Nk) (k) 
C) regression at collocation points [15] q" =u“, 


R, = R (a®), j=12..0 Ce G =ar 


m, = H(A) Je = (að ery" ee A 


where R,(q) is basis function j. Since it may be non-linear, RSM allows for non-linear propa- 
gation of uncertainty and may give a substantially different and more accurate result than LIN. 


If only linear basis functions are used R,(q)=q;, RSM becomes equivalent to LIN. The best least 


square approximation is directly obtained from Eq. 13 [19], 
-1 
C= (R'R) RH (14) 


Let RSM(r) utilize a complete set of mixed polynomial basis functions up to order r. Its least 
number (v) of collocation points grows rapidly with both the number of parameters (n) and 
polynomial order (r) [12], 
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(15) 


[ewt w(j,0)=1. 


n=2 n=5 n=10 n=20 
r=1 3 6 11 21 
r=2 6 21 66 231 
r=3 10 56 286 1771 


Table 1. Efficiency p =v for RSM(r), for selected polynomial orders r and numbers n of parameters. 


The distribution of samples may be improved with stratification, as in Latin Hypercube 
sampling (LHS) [18]. By dividing the sample space into intervals, or stratas representing equal 
probability the need for large ensembles is reduced. In LHS, each parameter is sampled exactly 
once in each of its stratas giving a generalized latin square [20]. This selection pushes the 
samples away from each other and distributes them more evenly. To illustrate the improve- 
ment with stratification, sample one parameter q~ NRM(0, 1). After division into m intervals 
of equal probability, samples are found with the inverse transform method described in section 
3.2 (Eq. 8). As seen in Fig. 2, the convergence improves dramatically. Still, even for m=100 
samples the second moment (left) varies noticeably. The convergence is generally poorer for 


higher order moments M k y as shown for k =4 (right). 


Figure 2. The second M (2) (left) and fourth M (a) (right) moments for stratified (solid) and brute force sampling (* ) of 
q~NRM(O, 1), compared to a fixed grid (dashed). 


In this case, it is questionable if 100 samples are sufficient to represent as few as four moments 


MM. The probabilistically evenly distributed fixed grid (dashed) converges more 
rapidly to the proper statistics. Despite the prevailing tradition, there is no absolute require- 
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ment of using a random generator to represent statistical information. Fixed grids are examples 
of deterministic sampling. Stratification provides an interesting intermediate type of sampling 
since it is partially deterministic — the strata are constructed deterministically but the samples 
within each stratum are generated randomly. The construction of a fixed grid requires focus 
on the most relevant features. To reproduce M a ® exactly, a very sparse grid or few 
deterministic samples are needed, 


(41.376 +0.325) q~UNI(0,1) 


iene 0(x4))  q~NRM(0,1) a 


If the problem at hand only depends on these moments, the exact solution will be obtained. 
The size of such small ensembles must be fixed, no matter how they are generated. Adding, 
or perturbing a single sample would modify the statistics substantially. 


4. Deterministic sampling 


Deterministic sampling (DS) of uncertain systems is a viable alternative to random sampling 
(RS). Instead of using random generators, specific DS rules are devised to generate appropriate, 
but still statistical (Fermi’s notation, see section 3.2) ensembles. A rudimentary example 
illustrates the principle: Assume a model y(q) depends on one parameter q with mean (q) and 


variance (5q *). To estimate the mean (y) and the variance (6 y °) of the model, the samples (filter 


parameters) 9) =(q) +4 (6q*) are appropriate since they satisfy the desired statistics, (q) b=) 


and (542) e=(5q7). The formula for qi) constitutes the sampling rule and qi) is the statistical 


ensemble containing only two model samples. By paying the computational cost of using more 
samples and improving the sampling rule, additional moments (6q*), k>2 or other statistical 
features can be accounted for. 


In deterministic sampling the model evaluations involve no approximations and are non- 
invasive. In many respects, deterministic sampling is constructed and optimized for quantifi- 
cation of modeling uncertainty: Minimal ensembles allow for evaluation of the most 
numerically demanding models. The model evaluations are exact and non-invasive to fully 
respect non-linear deeply hidden parameter dependences. Only vaguely known statistics of 
the model is approximated. 


4.1. Concepts of deterministic sampling 


DS does not per se specify the goal of sampling, e.g. given mean and covariance of the 
parameters. In the example at the end of section 3.3, the primary target was the joint pdf of the 
parameters. In section 4.2, the target is M Dq). In section 5, this will be complemented with 
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additional requirements. DS can also be utilized for direct evaluation of confidence intervals 
[12]. The targets of various DS methods may differ but the focus on the most influential 
statistical aspect and customization is shared. In stark contrast, almost without exception RS 
targets the joint pdf of the parameters and ignores the final utilization. Adaptation and fixed 
ensemble sizes provides the principal means to improve the efficiency of sampling. 


4.2. Propagation of covariance in the standard unscented Kalman filter 


The reference will be the specific variant of DS used for propagating covariance in what will 
be referred to as the standard unscented Kalman filter (UKF) [21-23]. The ensemble consists 
of 2n samples, or sigma-points, 


ge) =(q)+s-Vn-A,, AAT =cov(q), k=1,2,..n, s=4 (17) 


where A,, denotes the k-th column of A. The sampling rule is manifested in the square root 
calculation of the covariance matrix (A). As suggested [23] it may be found with a Cholesky 
factorization [19]. The square root matrix is not unique though — the Cholesky root is upper 
triangular and thus asymmetric. A more symmetric standard alternative is to evaluate the 
matrix square root in a canonical basis [24] Ug where cov(Uq) is diagonal. The canonical 
variations Usg ©”) will be unit vectors in the n positive and negative directions of the princi- 
pal axes of the covariance matrix, amplified by the marginal standard deviations and most 


importantly, Jn. . For many parameters with large covariance, the scaling with in may cause 
the UKF to fail since the scaling is not related to the variability of the parameters, only their 
total number. A possible solution to the scaling problem is provided by the scaled unscented 
transformation [25]. However, it is based on Taylor expansions and thus suffers from an 
approximation problem of the model. 


5. Sampling with conservation of moments 


One class of methods of deterministic sampling conserves a limited number of statistical 
moments. The model parameters are sampled to satisfy these moments and collected in 
ensembles, similar to how parameters are sampled to fulfill probability distributions in RS. 


5.1. Principle 


The constraints of satisfying statistical moments constitute an infinite system of equations for 


the samples 5q®. It can formally be viewed as sampling È ) of the joint pdf f (q), 
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m 


=(ðq) = fönf(a)adq 2 iia = (54;), 
Ty 

(69;159;2) = [aaaf (a)4q = Sa = (64.54%), (18) 
M yy 

(69;169;59;3) = [69;:59:264;3.f(4) 44 è Sae = (64.54 :264;5),- 


M y1 


Ill 
Ip 
Ill 


The infinite number of equations requires an infinite number of samples. However, it is 
implicitly assumed that relatively few moments are known and significantly influence the 
result of interest. Only a few moments then needs to be accurately represented by {oq}. 
Typically, (54;) and (5q;,5q;2) are estimated when models are identified [4,7]. In addition, the 
range M (9 or another higher diagonal moment can generally be determined from underlying 
physical constraints like stability. Clearly, any sampling rule must generate a fixed number of 
samples and create them simultaneously. The samples are consequently strongly dependent. 
One obvious sampling method is to solve Eq. 18 numerically for a sufficiently large number 
of samples 54, as in Eq. 16. Due to the strong non-linearities, this is quite difficult for a large 
number of moments but may be feasible for a few moments. 


5.2. The excitation matrix 


The UKF (section 4.2) utilizes DS with conservation of all first ((5q;)) and second ((54;154;2)) 
statistical moments. The invariance in its formulation allows for any additional ‘half’ unitary 
transformation A > AV, V:VV ' =I. This results in another equally valid matrix A, since 
AAT =AV[AV]' =AVV 1A! =AA". Since the transformation V is allowed and influences the 
result, the result of applying the UKF is not unique. The matrix V condenses this invariance 
and provides practical means to manipulate the UKF ensemble. A key feature of V is the 


absence of constraints on V TV . That makes it possible to stretch V ‘horizontally’ (as long as 
VV T =I). That corresponds to adding samples (sigma-points). The improved transformation 
U‘'SU (section 3.2) can be applied by also combining U with V. The square root of the 
covariance matrix will then read A =U "SUV instead of A=U TSV, 


Enmen = 44)" liem tU'SUV, V= Vm- V VV" =I, V.-1 


nxm” 


mx1 ~~ =0. (19) 

The samples ge are here collected in columns of the ensemble matrix X. The matrix 

cov(q)=AAT=UTS7U is diagonalized,S =eig(cov(q)), with the unitary transformation U 
AN 

[24]. The normalization factor/m is included in the excitation matrix V to satisfy the correct 


covariance, ([Z7)=AA7, just as the factor yn was included in Eq. 17 (the ensemble is here 
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expanded from n to m samples). The excitation matrix controls the sampling beyond the first 
and second moments, e.g. the range of the samples. Row k of the matrix SV can be interpreted 
as deterministic samples of the pdf ,(Sq,), assigned to canonical parameters in RS, see section 


3.2. All ensembles will be described with a unique excitation matrixV. 


The adopted transformation U Tous of S V distorts all higher moments than the second. 
This mixing effect is indicated by the index y(u TsuS~) defined in Eq. 11. To diagonalize 
large matrices cov(q) many efficient techniques have been developed. This should not cause 
any difficulties even for n ~ 1000, especially since cov(q) usually is either very sparse, or rank 
deficient for models with many parameters. 


5.3. Elimination of singular values 


The rationale for applying the reduction to be presented is that any model is derived from a 
limited set of experiments, resulting in a usually moderate rank of cov(q). If the number of 
parameters is large, it is thus often (nearly) rank deficient. The widely practiced singular value 
decomposition (SVD) [19] may then be used to reduce the excitation matrix and hence the 
number of samples. The most general form of SVD cannot be used here since it renders an 
asymmetric decomposition cov(q)=U'S?W, where UU T=UTU=WW"=W"W=I and 
S 20 jk Six 20- Different matrices U, W allow for decomposition of an arbitrary matrix. For 


the symmetric matrix cov(q), a symmetric SVD U=W can be found with the less general 
eigenvalue decomposition [24], according to the spectral theorem [11]. As cov(q) is positive 


definite, all its eigenvalues S fulfill the requirement of being positive. This is required to 


directly obtain a real-valued matrix square root A =U "SUV. 


The ensemble may now be reduced by elimination of singular values (ESV). Choose a threshold 
a and remove row r and column r from S and row r from U for all r such that, 


|S„|<a ‘max|Sy, 


, ax<<il. (20) 


Proceeding as in many applications of SVD, this reduction (indicated by tilde below) will not 
change the result significantly, if a is small enough. Accordingly, samples are eliminated using 
the alternative decomposition of the square root of cov(q), 


Mie SI) BV Veg Ae (21) 


nxn 


A^ 
Unfortunately, the less distorting transformation U TSU S~! of SV advocated in section 3.2 do 


not allow for r<n rows of the matrix V. The increase in distortion of M **2) 


indicated by 
wWuTsus")+WU") is less important for the intended use though. Signal processing 


models with large numbers of parameters are typically non-parametric and usually describe 
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samples of signals like impulse responses, or noise signals. The required LR property of the 
system then implies LP. The propagation of covariance is then linear and only the undistorted 
first and second moments need to be encoded. 


5.4. Correlated sampling of non-parametric models 


A major difference between parametric and non-parametric models is the dimensionality. A 
conceptual dissimilarity is that non-parametric models usually refer to correlated signals, 
rather than abstract model structures. The parameters may describe discrete samples of input 
noise [7], or an impulse response [6]. A common parametric pole-zero model may contain 20 
parameters, while a non-parametric model can be expressed in perhaps 1000 parameters. The 
ensembles of non-parametric models often need to be reduced drastically. 


Due to limited resolution, the correlation times of any signal or impulse response is finite. Their 
‘memory’ is thus finite so sample variations may be regenerated or repeated, as long as the 
time between repetitions exceeds the correlation time. This correlated sampling (CRS) provides 
efficient and accurate reduction of the ensembles. The minimal number of parameters n is then 
set by the correlation time of the model. Most importantly, the size of the ensemble becomes 
independent of the size of the model (the length of the signal). 


A finite correlation length t E€ N of any model ôx(t) is normally inferred from the decay of its 
autocorrelation functionC(t, T), where t denotes the lag and T refers to a non-stationary 


variation. Here, a global t will be defined through its / 2 norm and determined for a relative 
truncation threshold f (argmin returns the minimizing argument), 


| clr)= (afri )a{r =) p<<1. (22) 


If the model is expressed as a convolution 6x(t)=h (t) * w(t) of an impulse response h (t) and 


2 
T= mex arg min 
zi T 


x c(t)? - Pye) 


=t+1 


time-dependent white noise w(t) as in section 6.2, 
clen) Sar |r- 1t) non (uss) =n? Énna) n(t)=,[var(w), (w)=0. e 


By padding the model to an integer multiple of y 22t samples, it is always possible to choose 
an excitation matrix partitioned to block-diagonal form, 


£ y*y Dixy Ey 
A 0 cV. 0 te es m 
= yy vey ¥RY E aii = 
Vom = : + Ü Er c=], (24) 
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where eee is any allowed deterministic sub-ensemble. The factor c accounts for the change 


nxm" 


Sash IN 
from y samples of V yxy tO the m>y samples of V By violating the normalization constraint 


A A 
V axm 1m =m, the size of the ensemble can be ‘compressed’ from m to y samples by moving 


nxm nxm 
all sub-matrices cV ys} to the first block-column and skipping all zeros. The introduced constant 


c drops out as m > y, 


yxy yxy tyxy yxy 
V =| Py v s T Ley Lye dy 0 -I 
nxy | r Ynxy\|” ny) 5Y I FY iman (25) 


Accordingly, 


cov(x) lj-k|< 7/2 
cov(x), >(u"suvv"u"su) = i-k|> 7/2 . (26) 
f ik cov(x). a sal 
ik+ny” |n=arg min|j -k-ly| 


The consequence of violating the normalization constraint is that only a limited diagonal band 
of cov(x) is correctly reproduced. If anon-parametric model of a signal is propagated through 
a system model with impulse response h of correlation length ø <y /2 this will nevertheless 
not result in any error of var(i), as it is independent of all faulty elements cov(v) je 


| j-k | >y/220. To correctly evaluate cov(h ),,,, though, the size of the sub-ensembles Vg of 


uv 
correlated sampling must fulfill the stronger size constraint y>2(max(t, o)+ | u-v |). The 
symmetry of convolutions implies a corresponding result when the non-parametric model 


describes the impulse response h of the system, rather than a signal. 


5.5. Combining covariance 


A signal processing model generally includes both parametric and non-parametric sources of 
uncertainty. For instance, a device (parametric system model) may be fed with a signal 
corrupted with noise (non-parametric noise model). The question then arises how the two 
sources qj, Xy of uncertainty can be combined. For propagation of uncertainty through LP 


models, the combined covariance is given by the Gauss approximation formula [16], 


cov( y) = Deov| y), (27) 
k 
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(k)) 


where covly is the propagated covariance of q,, x,. This will seize to apply for non-LP 


models. There exists no general non-linear summation rule for propagated covariance. A 
method of summation can be given though, if different ensembles are combined as in RS. 


To combine ensembles of parametric (q) and non-parametric models (x), collect all parameters, 
A 

g= (q T xT)" and diagonalize the enlarged covariance matrix, cov(q)=U 'S*U. Build V with 

two blocks and use CRS (section 5.4) for the non-parametric model, 


A 


-1 


teg nxm 0 
0 V146-Vice : 
A am m A A 
Vinsky)x(m+o) 5 0 vV l+c- yxo aeS P ETE PT = (m + v) i (28) 
0 Vl+e-V 


The scaling y1 + ctl may cause a similar scaling problem as the factor in in the UKF (section 
4.2). Using extended excitation matrices these factors can be eliminated, 


z ‘a T 
| Braye aje] =c-l, c=max(m,v)>(n+/). (29) 


A disadvantage of this summation is that the same type of ensemble must be used for all 
parameters. Both alternatives combine the statistics of the two models non-linearly. The 
uncertainties are propagated and combined by evaluating the model for all samples and 
calculating the desired statistics, just as if the combined ensemble described one model. 


5.6. Selected ensembles 


The standard (STD) ensemble employed in the UKF (as defined in section 4.2) utilizes the 
perhaps simplest possible excitation matrix, 


A 


Verp = vn AL -Ln )» m=2n. (30) 


While the ultimate simplicity is its main advantage, the long maximal(!) range M ©) is its main 
disadvantage. 
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How far the reduction of samples might be driven is illustrated by the minimal simplex (SPX) 
ensemble, 


Vex =vyn+1-L cm = ai i, mn +1, (31) 


where the operator | performs classical Gram-Schmidt orthogonalization [11] and normali- 
zation of rows. The ensemble is constructed from half the STD ensemble, complemented with 
one sample 1,,,, to cancel the first moments. Since that violates the orthogonality of the rows 


of V, | must be applied to satisfy VV T=1.The high efficiency of the SPX ensemble is tarnished 


by its large skewness, or M ©. This may give considerable bias of propagated covariance for 
non-LP models, but is irrelevant for LP models. 


The binary (BIN) ensemble has minimal range to guarantee allowed samples. By varying all 
parameters with an equal magnitude of one standard deviation in all samples, the diverging 
factor Jn of the STD is eliminated. Its excitation matrix v pn is fundamentally constructed 
from a standard binary array, with the difference that the allowed levels are +1 instead of 0, 1 
(see rows 1-3 in Eq. 32). It is then complemented with supplementary rows obtained in two 
ways, by cyclic shifting and mirror imaging, 


+1 +1 1 1 1 +1 
+1 1 1 1 + 1 
+1 +1 +1 +1 -1 -1 -1 -1 7 
s K = cl 22 
ea a bm E) (32) 
+1 LaLa 
14 a <i 


Cyclic shifts are applied to all original rows except the first, by a quarter of their periodicity. 
Mirror imaging of a row is defined to change the sign of its second half and is applied to all 
original rows except the last two, and all shifted rows except the last. For instance, in Eq. 32 
row 4 and 5 are shifted versions of row 2 and 3, while rows 6 and 7 are the mirror images of 
rows 1 and the shifted row 4. The supplementary rows reduce the size of the ensemble 
drastically with a corresponding improvement of the efficiency. For n =20 parameters, the size 
drops from roughly 10° to 128 samples. That size is acceptable in perspective of the n + 1=21 
samples of the most efficient SPX. Eventually though, the number of samples will grow too 
large. The BIN can thus only be applied to moderately sized models. 
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By no means, this brief survey exhausts all possible ensembles. Many criteria for selecting the 
most appropriate ensemble can be formulated. Here, the first and second moments, parameter 
ranges and efficiency were in focus. 


6. Application — Modeling uncertainty of a dynamic device 


The task is to simulate the response of an electrical device such as an amplifier or oscilloscope, 
in the presence of non-stationary correlated noise on its input. An uncertain LR CT model of 
the device and its parametric covariance is usually found by applying system identification 
techniques [4] on calibration measurements [6]. Such a model of the system can be sampled 
into a digital filter and be described in the pole-zero form in Eq. 2. These standard steps will 
here be omitted. The system model will instead be assigned to a digital low-pass Butterworth 
filter, of order 10 and cross-over frequency fc=0.1fy, fy being the Nyquist frequency and 
described by parameters K, py Py «-- Piy Z Zy 249. The complete correlations of complex- 
conjugated pole (p) and zero (z) pairs are eliminated by a transformation from q=z, p to 
Re(q), Im(q)20, giving n=21 system model parameters, 


q=(K Re(z,) Im(z,)20 vee Re(p,) Im(p,) 20 vee Re(p,9) Im(p,))0) - (33) 


To be most general, the non-parametric input noise model is chosen to be correlated/colored 
and non-stationary. The noise parameter ôx, represents the noise level at time sample k. Its 


generating signal [7] is a Dirac delta function 6 ik? centered at time k. The response of a system 
with impulse response h will be dy;=6x;, - (h; * ô Jah j ` X,- In matrix notation, dy =h 18x, 
where A ,;=h ;_,. Hence, 


cov(y) = (5yéy")=h" (oxox" jh =h" cov(x)h (34) 


Since the response is linear in noise parameters, it is sufficient to only capture cov(x). 


6.1. Reference ensembles 


Traditionally, any method for uncertainty propagation is evaluated by comparisons with the 
default method of linearization [10,16], and brute-force random sampling (MC) [9] as state-of- 
the-art. There are several drawbacks of this approach. Linearization is a coarse approximation 
for LP models and MC suffer from the difficulty of modeling dependencies and low efficiency. 
An alternative is to construct finite reference ensembles (REF) and by definition let them 
describe the truth. Their primary advantage is that the finite size of the REF makes it possible 
to propagate the uncertainty exactly, using all REF samples. A more or less arbitrary REF may 
be generated randomly, like any MC ensemble. All requirements are also automatically 
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fulfilled since the REF is built of possible realizations. Also, the REF closes the loop as it makes 
it possible to compare ‘true’ and approximate samples directly on an equal footing (see Fig. 
8). Even though the samples differ substantially, the resulting modeling uncertainties can be 
similar. 


A plausible REF ôq; for the system model realized as a digital filter is created by randomly 
generating m samples of n parameters q, from uniform distributionsUNI(0, op), with g, listed 
in Fig. 3, top left. The joint pdf will have compact support [11], as required to guarantee 
stability. The mean is subtracted from all samples to remove the bias of the finite random 
ensemble, (ôq;)z =(6q;)=0, V j. The covariance of the REF will have a desirable more or less 
random variation for small values of m. If the REF samples are arranged in columns of a matrix 
Dron (as V) cov(g)ęggr=1/ mM: AAT, For m=31>n=21, the strong correlations will expose the 
methods to severe tests with significant transformations U, S. The mixing ¥ (Eq. 13) using 


transformation U TSU S~ was considerable, but less than for U ", see caption Fig. 3. For the 
chosen REF, the resulting variations of poles and zeros are displayed in Fig. 3. The obtained 
variation of the parameters defined in Eq. 33 can be quantified with an averaged correlation 
index and standard deviation (Fig. 3, bottom left), 


1 n-k 12 
an [tea | Set o? =var(q;). (35) 
Z 


A REF signal Ox; for non-stationary correlated noise may conveniently be generated from an 


autoregressive process (AR) acting on time-dependent zero mean white noise dw, ôx=g" dw, 
where g ij =8 j- ÍS the matrix of translated impulse responses g for the AR process defined by 


parameters a,. Assigning a square wave time-dependence, 


244544 = ôw, (6x5x") =3" (5wðw")g = 3" cov(w)z, cov(w) = diag| n(t) | 
z . 66) 
n(t) nizio 2 +6) af )1 x>(<)0 
lty p 0, x=0 


The exact REF result of modelling covariance of noise is given by combining Eqs. 34 and 36, 


On = (cov( {tr(cov(y)) = fte(a" z'diag| 7 [7 |g h). (37) 


An explicit realization of the REF for the noise model is hence not needed. Specifically, a second 
order system a=[1 -0.4 0.6] with time parameters {N, y, T, p}= {0.05, 03, 275). % 18} was 
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Figure 3. Top: Assigned variations (left) and resulting (z, middle, p, right) samples of the REF of the system model. 
Label P indicates the pole explored in Fig. 8. Bottom, left: Obtained variations o, (dots) and correlations € (bars) of 


parameters q (Eq. 33), with mixing (Eq. 11) wu lsus-)=0.22 (adopted) and Wu T)=0.39. Bottom, right: Impulse re- 
sponses h ((q), t) and g(t) and time-dependence n(t) (Eq. 36) of noise intensity. The correlation lengths Ap, Àg were 


determined according to Eqs. 22-23, for B=0.05. 


chosen. The impulse responses of the AR noise system and the system model, and the variation 
of the noise model are illustrated in Fig. 3, bottom right. 


The ‘true’ result given by the response for the REFs for the different test signals is shown in 
Fig. 4. The propagated noise variation oy differs substantially from the input square wave n 
(top left) and is almost opposite in phase, due to the response time of about fẹ", see delay of 
Ug (top, right and bottom). The signal distortion (uç) is strongly dependent on the input signal 
and decreases with increased regularity / differentiability. The propagated covariance o, has 


a more complex variation (top, right and bottom), as it is larger for the more regular Gaussian 
(bottom, right) than for the triangular pulse (bottom, left). 


6.2. Deterministic sampling 


The error of the scent and the standard deviation for the STD, SPX and BIN ensembles of the 
system model is displayed in Fig. 5, for all test signals. The low scent of the REF (left: thin, 
dotted) suggests the model is close to LP. Despite the relative errors are large they are quite 
small on an absolute scale. The SPX has the largest errors, for the scent as well as the variance. 
That is likely caused by its skewness being much larger than that of the REF. The BIN has the 
lowest errors and is thus the best approximate representation of the REF. 
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Figure 4. The mean ji, (dashed), the standard deviations Os y (solid) and the scent ¢ (thin, solid) for the REFs, for the 
different test signals (thin, dashed). The subscripts refer to the system (S) and noise (N) models. The variation of noise 


intensity is given by n(t) (top, left). 
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Figure 5. The errors of the scent -kgr (left) and the standard deviation 05—05 REF (right) of the system model (solid) 
for the STD (top), SPX (middle) and BIN (bottom) and the three test signals (thin, dashed). The correct gp (left) and 
Oc ger (right) are included for comparison (thin, dotted). The triangular and Gaussian signals are displaced for clarity. 


The errors might appear large, considering all ensembles are ‘correct’, i.e. correctly represent 
(typically) available accurate information (mean and covariance of parameters). The errors 
reflect ambiguities caused by the ubiquitous lack of information in signal processing, rather 
than inadequacies of DS. RS can only produce better results by making further assumptions. 


The result of applying the ESV and the CRS methods to reduce the SPX ensemble for propagat- 
ing the noise is displayed in Fig. 6. By choosing sufficiently low thresholds a for elimination of 
singular values (ESV) and £ for truncation of the correlation lengths (see Eqs. 20,22), the errors 
can be made arbitrarily low. As the reduction will decrease accordingly, there is a trade-off 


Deterministic Sampling for Quantification of Modeling Uncertainty of Signals 
http://dx.doi.org/10.5772/52193 


between accuracy and efficiency. For the chosen values, CRS is about twice as accurate and twice 
as efficient as ESV. In contrast to ESV, the number of samples for the CRS method is independ- 
ent of the number of noise samples. The computational cost thus increases linearly with the 
length of the noise signal for CRS but quadratically (approximately) for ESV. For ESV to be most 
efficient, the model covariance needs to be strongly rank deficient. That is not as unlikely as it 
might appear, since the model usually is derived from a limited amount of experimental results. 


Time (ff) 


Figure 6. The error ON- OW REF of propagated noise, for the ESV (section 5.3) and CRS (section 5.4) ensemble reduction 
methods, and the correct On REF (thin, x 1/10). The thresholds were a=0.1 (Eq. 20) for ESV, and B=0.05 (Eq. 22) for 
CRS. That resulted in m=142 samples for ESV and m=75 for CRS, compared to m=402 of the original SPX. 


The summation of the noise and the model covariance is illustrated in Fig. 7. The propagation 
of the covariance of the system model (q) is not LP. The quadratic summation rule (Eq. 27), or 
Gauss approximation formula [16], is therefore not applicable. Nevertheless, the low scent Ç 
(Fig. 5, left) suggests that both propagations are close to LP. The summation error (£) is hence 
finite, but quite small. It differs qualitatively from both contributions, indicating that the 
summation is non-trivial. 


Finally, the samples of one pole of the derived ensembles are compared to the reference 
samples of the REF in Fig. 8. The limit ( | z | =1) of stability is included to illustrate how close 
the samples are to be physically forbidden. The construction of the different ensembles is 
apparent, even though the transformation T =U T SUS“ distorts the scatter plots (sections 3.2, 
5.2), and tilts the principal axes (lines). The samples of the REF are almost evenly distributed. 
Only four samples of the STD, labelled p}, py, py Py deviate significantly from a dense central 
cluster, as described by the excitation matrix V srp (Eq. 30). It also is evident that SPX originates 
from half the STD. A small translation required to achieve the correct mean is discernible, while 
the Gram-Schmidt orthogonalization renders a minor rotation and distortion. The BIN 
contains comparable variations in all samples and thus has no central cluster and its samples 
are repelled from the principal directions (lines). The statistical differences to the REF refer to 
the shape of the joint pdf. Choosing the best ensemble is thus equivalent of selecting the most 
appropriate pdf in RS. The BIN seems to resemble the REF scatter plot the most, as verified by 
its low errors in Fig. 5. 
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Figure 7. Summation of covariance: Total (solid), system (dashed) and noise (dotted), for the three test signals (thin, 


dashed), with the error (e) of square summation (x 10) (Eq. 27). 
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Figure 8. The different samples (dots) of the pole marked ‘P’ in Fig. 3, of the reference (R 
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(SPX) and binary (BIN) ensembles. The limit |z| =1 of stability (solid, thick) and lines connecting the primary variations 
Pi Po, P3, P4 Of the STD as well as lines (dashed) to combined excitations of the BIN are included for reference. 
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7. Conclusions 


Deterministic sampling remains controversial [27] while random sampling has qualified as a 
preferred state-of-the-art method for propagating uncertainty. Both result in finite statistical 
[17] ensembles, which are approximate finite representations of the primary statistical models. 
Their sampling strategies and convergence rates are dramatically different. While determin- 
istic sampling humbly aims at representing the most relevant and best known statistical 
information, random sampling targets complete control of all features of the ensemble. Such 
detailed information is rarely known and must instead be more or less blindly assigned. The 
inevitable consequence is that critical computational resources are spent on propagating, at 
best, vaguely known details. The numerical power of modern computers is better spent on 
refinements of the signal processing model (longer time series, higher sampling rates, larger 
systems etc.). Refined methods of random sampling have therefore been proposed which either 
simplifies the model, or improve the sampling distributions. Compared to deterministic 
sampling though, their convergence rates remain low. 


It is easy to confuse deterministic sampling with experimental design and optimization [28]. 
Even though any sample could be a possible outcome of an experiment, deterministic ensem- 
bles represent rather than realize (as random ensembles) statistical distributions. Instead of 
associating a joint distribution to the parameters of an uncertain model, it is possible to directly 
represent their statistics with a deterministic ensemble. That would eliminate the need of 
interpreting abstract distributions and result in complete reproducibility. The critical choice 
of ensemble would be assigned once and for all in the calibration experiment, with no further 
need of approximation. 


The use of excitation matrices made it possible to construct universal generic ensembles. The 
efficiency of the minimal SPX ensemble is indeed high but so is also its third moment. While 
the STD maximizes the range of each parameter, the BIN minimizes it by varying all parameters 
in all samples. The STD is the simplest while the SPX is the most efficient ensemble. In the 
example, the BIN was most accurate. For non-parametric models with many parameters, 
reduction of samples may be required. Elimination of singular values (ESV) and correlated 
sampling (CRS) were two such techniques. The presented ensembles are not to be associated 
to random sampling as a method. They are nothing but a few examples of deterministic 
sampling, likely the best ensembles are yet to be discovered. 


It is indeed challenging but also rewarding to find novel deterministic sampling strategies. 
Once the sampling rules are found, the application is just as simple as random sampling, but 
usually much more efficient. Deterministic sampling is one of very few methods capable of 
non-linear propagation of uncertainty through large signal processing models. 
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Chapter 4 
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Analysis 


Alexey Mokeev 


Additional information is available at the end of the chapter 
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1. Introduction 


Analysis methods based on determining system performance specifications by step response, 
as well as indirect methods: pole-zero plot, magnitude response and integral analysis methods 
are applied in automatic control theory for performance estimation of linear systems [1,2,3]. 
However, in many cases the mentioned methods result in crude performance estimation of a 
linear system (filter) operation. Furthermore, direct methods of linear system performance 
specifications (settling time, accuracy, overshoot etc.) characterization require a huge amount 
of calculations being performed. 


Specification or estimation of signal processing performance criteria are usual tasks in 
frequency filter analysis. In some cases it is considered to be enough to examine a filter behavior 
at average statistical parameters of a useful signal and its disturbance. In other cases, for 
instance for robust filters, it is rather more complicated — one needs to determine the limit of 
variables for signal processing performance specifications at any possible input signal 
parameters variation. 


The author offers to use the filter analysis methods, developed by him on the basis of spectral 
representations of the Laplace transform, to solve efficiently the problem of signal processing 
performance specifications determination by frequency filters at different variations of input 
signal parameter [4,5,6]. The mentioned methods are based on using consistent mathematical 
models for input signals and filter impulse characteristics by means of a set of continuous/ 
discrete semi-infinite or finite damped oscillatory components. Similar models can be applied 
for simple semi-infinite harmonic and aperiodic signals or filter impulse characteristics, 
compound signals of any form, including signals with composite envelopes, as well as pulse 
signals (radio and video pulses). 
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The application of signal/filter frequency and frequency-time representations, based on 
Laplace transform, allowed developing simple and effective direct methods for performance 
analysis of signal processing by analog and digital filters. 


To simplify the task of analog and digital filter signal processing performance analysis the 
author offers two methods for performance express-analysis of signal processing by frequency 
filters using filter frequency responses based on Laplace transform: frequency and frequency- 
time analysis methods [7]. 


The frequency method from an indirect analysis method for signal processing, in fact, has 
transformed into a direct analysis method by means of Laplace transform spectral represen- 
tations. This method is the most effective in cases, where only two main performance specifi- 
cations: signal processing speed and accuracy — are required to be evaluated. 


The frequency-time analysis method is being applied in cases, where there is a need to evaluate 
signal processing speed and accuracy, as well as the history of transient processes in a filter, 
for instance, to control oscillation of transient process in a filter. It is suggested to perform the 
analysis by using frequency responses based on filter transfer function, dependent on time, in 
sections of input signal complex frequencies. 


In case of FIR filter an effective estimation of signal processing performance specifications can 
be carried out by using filter frequency response 3D analysis based on Laplace transform in 
sections of input signal complex frequencies, considering their change. To evaluate signal 
processing performance specifications for IIR filters one will need along with sections of 3D 
filter frequency responses to use sections of 3D signal spectrum on filter impulse response 
complex frequencies [5]. 


The issues about the application of the analog and digital filter analysis methods, developed 
by the author for signal processing performance analysis by frequency filters, are considered 
below. 


2. IIR filters analysis 


2.1. Signal processing performance analysis by analog IIR filters 


Let us consider signal processing performance analysis by IIR filters at semi-infinite input 
signals on the basis of analysis methods based on the Laplace transform spectral representa- 
tions. 


Three methods of frequency filter analysis are suggested by the author for the time-and- 
frequency representations positions of signals and linear systems in coordinates of complex 
frequency [5,6]. Let us consider the first two methods for signal processing performance 
analysis by frequency filters. 


Themathematical description for the generalized inputsignal and IIR filter at time and frequency 
domains for the first (item2) and the second (item 3) analysis methods, mathematical expres- 
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sions for calculating forced and free components of a filter reaction by the first method (items 
4,5), components of a filter reaction by the second method (item6) are given in the Table 1. 


Ne Name Expression Remark 


X=[X ply =[Xm,e/*'y, 


1. Input signal e f 
X (p)=Re|X | =, P=[Pr]y =[-Bn + Jn |yy 
t)=Re(G"e%) de es -jom 
7 aR G=[G Jy =[kme i "7 
2. Filter - 1 | : 
K(o)=Rel6 aar 9= [Pr =L-An* Mel 
5 Time dependent il a r ig 
` transfer function MP REE P-Pm M KAP: o-þwe ar 
4. Forced components y,(t)=Rel¥"e pt) Ý =diag(X )K(p) 
5. Free components y,(t)=Re(V'e%) V =diag(G)x (q) 
6. Filter reaction y(t)=Rel¥ (t)"e Pt) Ý (t)=diag(X )K (p, t) 


Table 1. IIR filters analysis 


The operation of the real part extraction on the right side of the expression in the items 1,2,3 
for X (p), K (p), K(p, t) is solved in terms of the complex coefficients G,, and p,, with no 


relevance to the complex variable p. 


The first method is a complex amplitude method generalization for definition of forced and 
free components for filter reaction at semi-infinite or finite input signals [6]. The advantages 
of this method are related to simple algebraic operations, which are used for determining the 
parameters of linear system reaction (filter, linear circuit) components to an input action 
described by a set of semi-infinite or finite damped oscillatory components. To analyze a filter 
it is needed to use simple algebraic operations and operate a set of complex amplitudes and 
frequencies of forced and free filter reaction components. In this case, there are simple relations 
between complex amplitudes of output signal forced components and complex amplitudes of 
an input signal (item 4 Table 1), between complex amplitudes of output signal free components 
and complex amplitudes of a filter impulse function (item5). 


The time-and-frequency approach in the second analysis method applies to a filter transfer 
function, i.e. time dependent transfer function of the filter is used [6,8]. In that case, instead of 
two sets of filter reaction components only one of them may be used. 


Analysis methods given in the Table 1 enable to reduce effectively the computational costs 
when performing a filter analysis by using simple algebraic operations to determine the forced 
and free components of a filter reaction to an input action as a set of damped oscillatory 
components. Therefore, the considered analysis methods for linear systems (filters) can be 
effectively applied for performance analysis of signal processing by frequency filters. 
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Let us consider a simple example of performance analysis of signal processing by a high-pass 
second-order filter relating to signal processing task in power system protection and automa- 
tion devices [9,10]. The filter is used to extract a sinusoidal component of commercial frequency 
and eliminate disturbance as a free component of transient processes in a control object. In this 
case, a change of a useful signal initial phase is acceptable. 


All the initial data and dependencies which are necessary for the analysis are represented in 
the Table 2. IIR filter parameters are specified, the mathematical description of an input signal 
with specified sizes of changing for useful signal and disturbance parameters affecting their 
spectrum is given in the Table 2 as well. 


The impulse function of high-pass second-order filter contains a delta function of Dirac which 
is used for determining complex amplitudes of forced components when defining K (p) by the 
impulse function (item 3 Table 2) and cannot be applied for determining complex amplitudes 
of filter reaction free components (item 5). To simplify the analysis the delta function can be 


represented as an extreme case of the exponential component ae“ at a — [6]. 


The analysis results should ensure the following performance criteria of signal processing by 
a filter: 


1. a filter settling time should be less than 30 ms at 5% acceptable total error of signal 
processing at any value of disturbance parameters within the specified range, 


2. an acceptable error at frequency deviation of useful signal from the nominal value of 50Hz 
within the range +5 Hz should not be more than 5%, 


3. an acceptable overshoot should not be more than 10%. 


As it follows from the Table 1, simple algebraic operations are applied to determine complex 
amplitudes, as well as forced and free components of a filter output signal. 


When using Mathematica, Mapple, Matlab, Mathcad and other state-of-art mathematical 
software for determining forced and free components of an output signal it is necessary to 
specify only complex amplitude vectors of an input signal and a filter impulse function, as 
well as complex frequency vectors of an input signal and a filter. In this case all the necessary 
calculations, related to a filter analysis, would be carried out automatically. If it is needed to 
determine complex amplitudes of a filter impulse function at specified transfer function the 
ready-made formulas may be used [6], which can be easily applied in the mathematical 
software mentioned above. 


All the examples in the present chapter are given using the mathematical software Mathcad. 
Mathcad was chosen due to pragmatic considerations related to assuring the maximum 
visibility of the examples for filter analysis, as in Mathcad mathematical expressions are given 
in the form, closest to universally accepted mathematical notation [11,12]. 


An example of a filter computation using Mathcad at the specified filter parameters and the 
following input signal parameters: X m,=Xm,=1, w,=27t50rad/s, p,=0, 6,=60 s* is given on 
the Figure 1. 
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Ne Name Expression 


x(t)=Xm,cos(w,t-9,)—Xm,e ™, 


X=[Xme/" xme], p=[ia, -bf, 


1. Input signal 
Hb=X m,/Xm,=0+1, w =27(45+55), p, =0+27, B>=2+200 
pcos(p,) + w,sin(,) 1 
X(p)=Xm, pau -XM 545, 
= kop? = = il EK 
Ke git)=ky6(t) + RelG,e""),ky=1, 206, 


2. High-pass filter 
G=[k,e /*']=[-424, 5e 9342], q=[-a; + jw;]=[-165, 9+ j117, 1] 


v-a Heer e oes 


4 Forced components of 0 XaLK(-B)} LX 2K(-B) LY ot 
` complex amplitudes | = 044w2e 7” Be~)" | 
=Xm 
"| 100?-w72+ j166, 66w, 1007+8)-166, 668, 


y,(t)=Rel¥"e ?*)=Re Rely e ®t 4. ye), 


4. Forced components 


Yi T| a iat 
4 al 


ylt)=V, q(t) + yalt), Yar (t)=RelY je), y,,(t)=RelY e t) 


Complex amplitudes of free . _. ; p;cos(@,) + wsin(o,) 1 
5, V =[V ;]=[X (0,)G -xm -u 
aoriponsnks LV J=[X(0)G,] 1 p+ w? 2, Fb 
6. Free components y,(t)=Re(V ,e”") 
7. Error e(t)= Y(t) + y(t) 


Table 2. IIR filter analysis 


For determining or estimating performance specifications of signal processing by the investi- 
gated IIR filter one need either to improve the software (Figure 1) or to reduce the amount of 
calculations by simplifying the analysis task. Let us consider the second option first. 


The easiest operation is to define the error level in signal processing by a filter at frequency 
deviation of useful sinusoidal signal within the range +5 Hz from the nominal value of 50 Hz. 
This error, as it is known, may be determine by an average amplitude-frequency response of 
a filter. In this case the value of a filter amplitude-frequency response in the areas of frequency 
27(45+55) rad/s is between 0,95 and 1,038. Thus, the filter meets the signal processing 
performance requirement mentioned above. 


A filter settling time can be defined by total damping of a free component q} and a forced com- 
ponent of exponential disturbance q, if the last component was not eliminated by the filter till 
the necessary level. Time q} and T, can be defined according to the Table 1 (item 6 and item 4). 


A damping time of disturbance free component tT, and forced component t, to the required 
level of 5% may be determined on the basis of the expressions given in the Table 2. 
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ORIGIN:= 1 jey-l 
Input signal xc -iy pl:=(j-2n-50 60)! N := length(X) 
> T 
IIR filter G= (424.4054 0) p = (-165.9+ j-117.1 0)" M := length (G) 
iw} X (x), iM] & ©; 
1 METOD XI(p) == > +— KI(p) == E + — | + 1.206 
22 [P-P p-(pi); 22 |p>pi p-(p); 


Complex amplitudes: y:=diag(X)-KI(pl) Yf:= diag(G)-X1(p) 


yp(t) := Rel yl Pl +) Yp(t) = Religon) elt) yf(t) = rdvi et) y(t) =yp(t) + yO Elt) = yf + Yp(t), 


1 
y(t) 
yp (t) 
F(t 
yf (t) a J 
e(t) 
Yp(t)2 
-1 
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 
t 
mM [ G (A), Be 
—(p-— p-\p/.|t 
2 METOD K2p,t) => : lied (p Ppt i lie [)] + 1.206 
22 | p-pi alp). 
i=1 i 


plit) YMQ(i, t) = |Y2i,t) PePli) ys(t) => y2i,t) 


i=1 


YXi,t) =(X))-KAp1,,t) y2(i,t) = nd vai,tye 


ys(t) 
y2(1.t) 
y2(2.t) 


YM2(1,t) 


0 


0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 


Figure 1. IIR filters analysis using Mathcad software 


1 0,05 1 0,05 
T4 In 779 


1 ; 
Oy kı |X (P1 )| Bo : H2 |K (-82)| 


(1) 


Variables q} and t, depend not only on filter parameters which are constant, but also on signal 
parameters. Let us assume the worst option p,=1, p,=0, w,=2750rad/s, taking into account 
the particularities of the controlled object [9,10]. Thus, let us consider the dependence t,and 
T, from fp. 


A presice settling time t can be determined through the total error of signal processing e(t). 
In case of t2 tthe condition | e(t) | <e,,,should be performed, when &;,,,=0, 05. 
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Let us consider an estimation of total damping t by the specified values of t, and t, 5 %. In 
this case T<T, + T>. If the filter is designed in a correct way, then | y,,(t) | < Eim at #27, that is 


when disturbance is eliminated to the specified level by the moment of the end of transient 
process ina filter, then t = q}. An estimation of a filter settling time can be performed with some 


conservative value on the basis of a sum of modules of free component envelopes of transient 
process in a filter and disturbance forced components [13]. The dependence t from f, can be 


quite easily determined by an insignificant improvement of the program on the Mathcad 
example, represented on the Figure 1. 


Dependencies t, tT, and Tt, depending on the value of exponential disturbance damping 


coefficient 6, are shown on the Figure 2. 


0.03 


0.02 


0.01 


Figure 2. The dependence of a filter settling time from damping coefficient B, 


Incase of 6).=2+35 s7 the initial level of a disturbance forced component is below the acceptable 
error, so | Yal S£im and t,=0. Then a filter settling time is mostly defined by damping 
transient process of its own in a filter, in other words by the value t,. Within the range 
B,=74+125 s” damping of a disturbance force component is longer than damping time of 
transient process of its own in a filter, that is T,>t,. At B,<107 s” a filter settling time T is less 
than values q} and t,, and at B,>107 s” is longer than any of the values mentioned above. It is 


due to plus-minus signs of filter reaction components, defining an error of signal processing, 
as well as to values of the components complex frequencies. 


An overshoot level in a filter can be determined by the program improvement on the Mathcad 
example, shown on the Figure 1. 


The performance analysis results for signal processing of the investigated IIR filter are 
represented in the Table 3. Performance specifications determined by using one traditional 
method - by step response of a second-order low-pass filter are given in the Table 3 as well. In 
this case the description for the low-pass filter was obtained on the basis of the investigated 
second-order low-pass filter by applying a well known frequency transformation [14]. 
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Ne Name Step response Direct estimation 
1. Settling time, s 0,0172 0,0275 

2. Maximum error in the steady-state mode, % 0) 5 

3. Maximum overshoot level, % 1,17 11,81 

4. Additional error at a frequency variation of an useful signal - 5 


Table 3. Signal processing performance for a second-order high-pass filter 


As it follows from the Table 3, in the considered example there are some substantial differences 
in the performance specifications, gained by the traditional method and on the basis of their 
direct determination. 


2.2. IIR filters analysis at dissemination of time-and-frequency approach to transfer 
function filter 


As it follows from the Table 1 (item 3 and item 6), estimation of functioning performance for 
filters can be performed at dissemination of time-and-frequency approach to filter transfer 
function, in other words by using time dependent transfer function of a filter [5,6]. 


In that case, comparing to the first method where two groups of filter reaction components are 
using — forced and free components, only the first group of components is using. The infor- 
mation about the transient process is containing in the time dependent complex amplitudes 
of a filter reaction Y(t). In the case of t > œ the mentioned set of complex amplitudes will be 
equal to complex amplitudes of forced components Y (t) > Y. 


The necessary dependences for determination of output signal components of a filter are 
represented in the Table 1, an example for high-pass filter analysis at the specified parameters 
of an useful signal is given on the bottom part of the fig.1. 


The advantage of the considered method is connected to determination envelopes for every 
component of a filter output signal, based on which the total envelope of a filter output signal 
and a variation law of initial phase can be defined. This information can be effectively used 
for performance analysis of signal processing by frequency filters. For instance, when deter- 
mining an overshoot level (oscillativity) of a transient process in a filter. 


2.3. IIR filter express-analysis method 


It follows from the Table 1, that quality indexes estimation for filter operation can be carried 
out on the basis of interim calculation results —K (p) and X (q), that means - based on spectral 
representations of signals and filter impulse functions in complex frequency coordinates. 
Another, not less effective approach, is related to the usage of the interim results of the second 
analysis method — the transfer function K(p, t) which is dependent of time. Thus, the 
application of filter frequency characteristics and a signal spectrum in complex frequency 
coordinates increase significantly the effectiveness of using the frequency methods of per- 
formance analysis for frequency filter operation [7]. 
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The express-analysis methods for filters, including performance analysis of signal processing, 
were developed based on investigation of 3D and 4D frequency responses [7]. It is enough to 
consider the sections p= jw and p=-y of 3D frequency responses K (p), as well as the section 
p=-a,+ jw, of a input signal spectrum according to the Laplace transform to estimate the 


settling time and accuracy of signal processing for the example given on the fig.1. 


The express-analysis methods mentioned above can be effectively applied for FIR filters as 
well, the detailed explanation will be given further in the present chapter. 


2.4. Digital IIR filters analysis 


Under the definition of digital filters in the chapter discrete filters are ment. In many cases it 
is justified, for instance, in cases of using microcontrollers or digital signal processor with high 
digit capacity and especially for microprocessors with support for floating-point operations 
[15,16]. 


When using discrete filters their analysis has a lot of similarities with the analysis of analog 
filters-prototypes. There is a small difference only when it comes to transition from images to 
originals. The main expressions for determining components of a digital filter output signal 
when injecting on the filter input a signal as a set of discrete semi-infinite damped oscillatory 
components are given in the Table 4. 


An example for digital filter analysis as a continuation of the example of the analog filter- 
prototype analysis (fig. 1) is represented on the fig. 3. The mathematical description for the 
digital filter was obtained by the method of invariant impulse responses at the discrete 
sampling step T =0, 0005s. 


Ne Name Expressions Remark 
—Re(XT. ) X=[Xme 2” |y, P=[-B, + jw 
x(k)=Rel(X 'Z(P, k)), In n: P=[-B, + [On] y 
1. Input signal x(2)=Re{X] z | z=eP",Z(p, k)=e”, 
Z-Z, ÍN : : 
T- discrete sampling step 
P - -jom 
g(k)=Re(G"Z(Q, k)), G=[k,e "yy, 
2. Filter ` =[-a_+jw.],,, 
K(z)=Re "| | a=[- am + Mina 
T Em IM z=ed 
Time dependent transfer a PmkT -k k , 
ne K(z, k)=Re eal | K(z,k)=) g(z7 
function Z= M i=0 
4. Forced components y,(k)=Rel¥'Z(p, k)) Ý =diag(X)K (z), 
5. Free components yy(k)=RelV 'Z(q, k)) V =diag(G)X (z) 
6. Filter reaction y(k)=Rel¥ (k)'Z(p, k)) Ý (k)=diag(X )K(z, k) 


Table 4. IIR digital filter analysis 
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T := 0.0005 z1=@!T SPT zs(ps ,k) = ET k:=0. = 
TÄ Xyz (x) 2 pM | Gz (9; 
X2(z) = > + = Kz) = — + = | + 1.206 
A e 
Ydl := diag(X)-K2(z1) Yd2:=diag(G)- X20) 
ypd(k) = revar"-zs(pt, 9) yfd(k) = re var" zs(p 19) yd(k) = ypd(k) + yfd(k) 
1 Pf 
pO 
wO 
#09 
piw 0 
yow 
ga 
24 RS 
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 
t,t,t,k-T,k-T,k T 
Figure 3. IIR digital filter analysis using Mathcad software 
Ne Name Expressions Remark 
x(t)=RelXTe P(Ct-t)_ X'Te Picit), X =[X ny =[Xm,e/”"]y, 
X '(p)=Re x| l | : X'=diag(X)e Pt), 
P-Pn In 
= =[-B, + jw, ]y, 
1. Input signal X"(p)=R x 1 P=[Pr]y =[-Bn + jon]y 
PITAR P= Pn In" P=diag(p), 
-Pt, -Pt, t=[tn]v t =[t]y. 
Xx(p)=Relx"| | +X £ 
= Pn dn P—Pp In C=[1]y 
t)=Re(GTe%), -jom 
l a 6 =[G ly =lkne "le 
2. Filter 1 | 
K(p)=Re G E P q =[Pm]m = -am + jWm]m 
Time dependent transfer Sl 1-e vo) | f > 
A = K(p, t)= d 
function K (p, t)=Re|G PPr Iu (p, t)=|g(t)je "dr 


4. Forced components y(t)= RelýTe Pict) Te. (Ctt ) 


¥'=diag(X ')K (p) 
: . ; V IG mX Pm)n]m w’ 
5. Free components y(t)=Re| > yinte ao) yore a oe r 
i E V =[GnX (mada 


6 Filter reaction 


Ý '(t)=diag(X")K (p, Ct-t') 


Table 5. IIR filter analysis at compound finite signals 


Y (t)=diag(X )K(p, Ct-t) 
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2.5. IIR filters analysis at finite signals 


Performance analysis of processing finite signals by IIR filters as a set of damped oscillatory 
components with finite duration may be performed on the basis of dependencies for IIR filters 
at semi-infinite signals [5,6]. 


All the needed expressions were obtained on the basis of the expressions from the Table 1 using 
a time shift and the principle of additivity. 


Let us consider the IIR filter analysis at compound input signals as a set of sequentially adjacent 
finite signals [17]. The calculation of IIR filter reaction for this case is represented in the Table 5. 


In this case every component of an input signal in a general way has a different shift and a 
different duration, where Vv“) — n-th matrix column V. 
The expressions represented in the Table 5 can be significantly simplified, if the IIR filter 


analysis at a finite signal, for instance, at injection a finite signal on its input from N number 
of components with equal duration and the same time shift [6,17]. 


An example of filter calculation, analogous to the example on the fig.1, but using an input 
signal with finite duration is given on the fig.4. 


The dependences given in the present section can be effectively used for not only analysis of 
passing through IIR filter one or another finite signal, but also for performance analysis of 
signal processing by filters. 


ORIGIN:= 1 j=yo 
Input signal x:=(1 -1)' pt:=(j2n50 -60)' t1:=(0 0)" t2:=(0.0448 0.0448)"  N:=lengh(X) i:=1..N 
N 
XI xotir) xXi,t) srd xpe" of - 4) - xy eof = 2) xt) = > xXi,t) 
j0342 .\F oe 
IIR filter = hazas pi o) p = (=165.9+ j-117.1 0)" M == length (G) 
MJ] S - (p= (9 -[p-to),f 
Kp, t) =F L li-e ( ed], = Fee [ J + 1.206 
2 P~Pi pP- (5). 
i=l i 
YXi,t) = (X) KAp1;t = t1;) YXi,t) =(X1))-K2(pl,.t- 12)) 
N 
y2(i,t) = adyai.n of.) - vain o-n) y(t) => y2(i,t) 
i=l 


LATAA 
“MV 


0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 


Figure 4. IIR filter analysis at a finite signal 
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For digital IIR filter analysis at injection finite signals on their inputs analogous mathematical 
operations are applied. All the necessary dependences may be obtained on the basis of the 
formulas from the Table 4 [13]. 


3. FIR filters performance analysis 


3.1. Particularities of the analysis 


Mathematical description for FIR filters can be obtained on the basis of the IIR filter description 
(Table 1) by using twice as many of filter impulse function components. 


Ne Name Expressions Remark 
; z X=[X,], = Xm e /% 
1. Input signal x(t)=RelTeP'), x(p)=RelX| | ce l 
a P=[Prly =[-Bo + Jn] 
g(t)=RelGTe% —GTe Ut), G=[G nly kne | 
K(p)=Rel 6] l ) G =diag(G)e%, 
m IM ; 
2. FIR filter "ERT 9= [0m] =[- 4m + Jnl 
K>(p)=Re\G"] ———e "| ], =diaa(g) 
2 P-Pm m Q=diag(q), 
at T=[T nly. C=l11u, 
K,(p)=Re(é Ts] mim 
P=Pm Im K(p)=K,(p)-K-(p) 
3. Forced components y,(t)=Rel¥ Te pt) Y =diag(X)K (p) 


V =diag(G)x (q), 
V'=diag(G )X (q), 
Ù =diag(X )K,(p), 


D'=[X K3(Pn)mly m 


Yo{t)= y(t) + y4(t) 


4. Free components ys(t)=Re(VTe%—V Te cT) 


y,(t)=y,(t)— Re(UTe pt -5 Ù Te Pea) 


5. Filter reaction y(t)=y,(t) + y(t) 


-(p-p,,)t -(P-Pm)lt-T m) J 
1 1l-e m m: m 
G | P=Pm 


1-e 
P-Pm Jm” 


| l K(p, narel6'| 
m 


ply 


Time dependent  K,(p, t)=Re 


` transfer function 7 
K(p, t)=K,(p, t)-K,(p, the 


Y (t)=diag(X)K,(p, t), 


7. Filter reaction y(t)=Rel¥ (t)"e Pt -Yý (tye Pt) ba ar 
Y (t)=diag(X )K>(p, t) 


Table 6. FIR filters analysis 


The additional components have the same set of complex frequencies and differ by time shift 
and values of complex amplitudes in a way to ensure the finitude of a filter impulse charac- 
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teristic [5,6]. According to this approach to mathematical description, IIR filters are special 
cases of FIR filters. 


The input-output dependences for FIR filters can be obtained on the basis of analogous 
dependences of FIR filters by using time shift and principle of additivity operations [6]. 


Comparing to IIR filters, FIR filters have finite duration of transient processes of their own, 
which are defined by a filter length. This to a certain extent simplifies performance analysis 
for signal processing of this type of filters, especially when using the suggested express- 
analysis methods for signal processing performance by FIR filters. 


3.2. Analog FIR filters 


Basic expressions for FIR filter analysis at injection on the filter input a set of semi-infinite 
damped oscillatory components are given in the Table 6. 


Due to characteristics of FIR filters, among forced and free components in an filter output signal 
there is the third group of components y,(t), which is conventionally referred to free compo- 


nents in the Table 6 [6]. 


Ne Name Expressions Remark 

X =[Xaly: P=([Pr]ys 
X'=diag(X)e Pt), 
P=diag(p),C=[1]y , 


t=[th]yt =[thlv 


1. Input signal x(t)= Rel Te PICt-t)_ Te Pict-t}) 


FIR filter: G= nli =al 


Ġ'=diag(Ġ)e 97, 


impulse function, 


` time dependent k il sas | pes 
K t)=Re|G =di .T=/T 
transfer function ap, t)=Re P-Pm m” Q=oiagig [mlm 
K(p, t)=K,(p, t)—K,(p, tle P" 
Y ,(t)=diag(X)K,(p, Ct-t), 
ylt)=Rel¥ (te PCy (tye Ptt) ¥(t)=diag(X)K,(p, Ct-t’), 
3. Filter reaction . ý i 7 
-Y,(t)'e P(C(t-T,)}+t) $ Y,(t)"e P(C(t-T,)-t ) Y (t)=diag(X )K,(p, Ct-t), 


Y ,(t)=diag(X )K>(p, Ct-t) 
Table 7. FIR filter analysis at input signal as a set of finite components 


Basic expressions for FIR filter analysis at signal injection on a filter input as a set of damped 
oscillatory components with finite duration are represented in the Table 7. The calculation of 
a filter reaction is given in the Table 7 using only the second analysis method for the case when 
duration of all the components of a filter impulse function is equal T,. The filter analysis based 


on the first method is represented in details in the author's papers [6,18]. 
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Let us consider an analysis example of FIR filters which are used in one of the most perspective 
intelligent electronic devices (IED) - Phasor Measurement Units (PMU) [19]. 


A brief description of a basic algorithm for PMU signal processing on the example of an analog 


system-prototype is given in the item 1 of the Table 8 [20]. 


Ne Name Diagram/expressions 


1. Block scheme of algorithm 


Semi-infinite signal 

z(t)=Re(zZ"e") or z(t)=0, 5(Z%e"t +. Z"e"*), 

Z=[2Zo 21 22 2. Zeal", 

2. Signal description z(t) r=[~Bp jw, J20) ig Mon, 
Z,r— are complex — adjoint vectors 
Signal as a set of finite components 


2(t)=0, 5(ZTe R(Ctt)_Z'te RICE) | FT Ř(Ct-t)_ 2'te Ř(Ct-t)) 


R=diag(r) 
x(t)=2e z(t), 
3. Input signal of a filter At semi-infinite input signal device 
x(t)=Z7e (r-C jut )t 4770 (F-€ jot Jt 
t t 
4. Algorithm X,(t)= | z(r)e /°"g(t—1)dr= | x(t)g(t-t)dr 


T; 


G=[80.48e 14273 37.93e 105887]! G'=diag(G)e? 
q=[-22.99 + j62.30 -23.26 + j186.9]', 
=(T, AET e 
5. Average FIR filter T=[7; 11], T,=0.051c, 


g(t)= RelGTe _GTe at-T)) 


epee 
P-Pm Im P—Pm M 


K(p)=Re 


Table 8. IED algorithm 


An input signal of intelligent electronic devices is represented by a set of complex amplitudes 
Z and frequencies r, as well as time parameters when using a signal model as a set of finite 
components. Let us constrain the signal models to one model only as an exponential compo- 


nent, useful sinusoidal component of commercial frequency w, (nominal value is w)=2750rad/ 


sec) and higher harmonics. More complicated signal models are considered in the papers [6,20]. 
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An average FIR filter K,(p) should isolate the constant (y(t)= X 1 at w =w) or low-frequency 
component (}(t)= X (t) at w, #p). The filters should suppress higher harmonics and a damped 
oscillatory component with the complex frequency p=—f) + jw. 


The considered filters should have low sensitivity to a change of damping coefficient fy in the 
range from 10+200 sec’ and the frequency w,=271(50 + 5) rad/sec. The acceptable static error of 
signal processing should not be more than 0,5%, and the acceptable dynamic error at t2T, 
should not be higher than 3%. 


FIR filter analysis is performed at input signal of a device as a set of semi-infinite or finite 
damped oscillatory components according to the algebraic expressions from the Table 7 and 
the Table 8. 


An example of FIR filter analysis using Mathcad at compound input signals as a set of 
sequentially adjacent finite signals is given on the fig.6. 


= oi 
Function AXI, X2,p,t1,12, [xr Dot- — x20). — 129] 

G sip G zis 

w(G.p.p.t):= harto] li Lele o)] 
BaP. P-P 

ORIGIN:=1 j= @l:= 20-50 o0:=2750 B0O=20 ri:=jol r2:=-B0 pO=jo0  Bl=5  do=j2m2 
ti=0 2=02 B:=06 4:=0.7 G18 16:=3.6 

i n " is Š ioe WX. 
Input signal device z:=(0.36 95" -0.1 o.0se 55" 25 25 0 26 FOS a6 FOS pg FOS" gg. HOS" g.4g i058) 

A T T 
ris (rl brl Srl ri 12 0 ri -Bl+ jol rl rl+do rl- do) ds( dd 2 2 B 4 4 5 15) 


Rih 2 VB BAS 15 6 6) Ki=length(Z) k:=1..K 


K Pi A D 
z(t) = ; X laz ed tpt] + Jaa, Per = OER | 


Input signal filter Xk 52k P15 PO 2, = (r) = p0 
a4 us xx ply(t2y-thy) sett A(X),.) p2y(t2ythy) a 
KORE > eo Xe Ply stl stat] + AM ee PA t 
k=1 


i \T 
2732 j 0.588" 
54.2732 379336705887) 


FIR filter G:=(s0.48326 p = (=22.9881+ j-62.3049 —23.2599+ 5.186.894!  T1I:=0.051 TF:=(T1 TI)? 


Gl:=diag(G-e? T! M :=lengh(@Q  m:=1..M 
1 M 1 M 

Kp.) = 5 2 W(G p Pm:P-t) K%p,t)= 5 ` W(G1,,.Pm-P.t~ TI) 
“m=1 “m= 

a y Piety) (F) Pet) 

X= Xpe x2, (xee 


K 
yl(t) = > (Yx; Ki(pl,.t~t1,), XI Ki(pl,.t~12,),Ply tly .t2,.t) ~ HX KAP t= tlp) X1,-KApl,.t~12,),pl, tly + 11,2, + T11) 


k=1 
K 7 = 
y= Yo GIEN KI(p2t = t1) X2,-Ki(p2, .t = 2) Patet] = a ( xX) KA p2,et = t1) XZ KA P2 ot = 2): p21, + TL2 + Tial] 
k=1 
y(t) = 0.5(y1(t) + y2(t)) Tz:= 0.03 zát) := if(t > Tz, a(t — Tz),0) 


Figure 5. FIR filter analysis using Mathcad software 
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Each of five sets of finite components represented comply with a particular power system 
regime. The first set is related to a normal regime of a power system and is represented by 
sections of sinusoidal component of industrial frequency and higher harmonics. The second 
set of finite signals corresponds to an accidental regime and contains of finite sinusoidal and 
exponential component, the third set shows no-current regime, the fourth set is connected to 
a change of PMU input signals envelope due to automation operation, the fifth set represents 
a swings regime. 


A filter processes a complex signal x(t), formed after multiplication of PMU input signal z(t) 


by a referencesignale jot to shift the signal spectrum to the left for the formation of orthogonal 
components of complex amplitude of sinusoidal signal with industrial frequency %4. 


A plot for PMU input signal z(f)and a plot for a module of filter output signal | y(t)! taking 
into account filter group delay time are represented on the fig.6. As it follows from the fig.1, 
the plot | y(t) | is close to the envelope of sinusoidal component of PMU input signal. 


It appears from the plot that the investigated filter has a feature which is connected to the 
absence of overshoot (oscillation) of transient process in the filter in its traditional definition, 
in other words at a stepwise growth of a signal. and presence of overshoot at a stepwise 
reduction of signal. 


In fact, due to particular characteristics of impulse function, from the traditional point of view, 
an overshoot is absent in the considered FIR filter, as at the end of transient process in the filter 
signal behavior is close to aperiodic process. In this case oscillation is noticeable on the initial 
stage of transient process. 


Analysis of FIR filter and signal processing algorithm in total is carried out only at fixed values 
of input signal parametres in the example. With some further ordinary improvement, analog 
to the example on the fig.1, it is possible to determine performance specifications for FIR filter 
signal processing at any variation of useful signal and disturbance parameters. However, it is 
much easier to use the specific express-analysis methods. 


3.3. Express-analysis methods for signal processing performance 


Let us consider an example of express-analysis of signal processing performance of a FIR filter, 
which mathematical description is given in the Table 7. In addition, let us consider performance 
analysis for signal processing of a filter with the impulse function g,(t)=9,(T,-t) (Fig.7). 


To check the adherence to the mentioned conditions, it is enough to consider amplitude- 
frequency response of the filters in complex frequency coordinates. 


The amplitude-frequency response for the first and second filters is given in the Fig.8. The 
multiplier ef" at B#0 allows for attenuation of forced damped components by the moment 
of the transient process ending in the filter. The speed of the FIR filter is determining by its 
length T; in the case of noise damped components are suppressed to the level of the acceptable 


dynamic error during the time, that is equal to the filter length. 
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Figure 6. Impulse characteristics of the filters 


|K(-B + joje 


Figure 7. Amplitude-frequency response of the filter 


To check the compliance with the requirements for filters to the specified quality indexes of 
signal processing for the specified input signal, it would be enough to consider two sections 
of 3D amplitude-frequency response: in the sections p= j27f (Fig.6) and p=-—f + jay (Fig.8). 
As it follows from the Fig.6, the examined filters have the same amplitude-frequency re- 
sponse in the section p=j27f and ensure the required quality of useful signal processing 
and suppressing higher harmonics. Absolutely different situation appears in the case of 
eliminating by filters a damped oscillatory component with frequency pọ=-fo+ jwg. 


KP" 


0.04 


0 50 100 150 


Figure 8. Frequency response of the filters in the section p=—B + jw 
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The filters 1 and 2 in the section of 3D frequency response p=-f + jwg (Fig.7) have different 


characteristics, and the second filter does not ensure compliance with the specified require- 
ments to suppress the component of an input signal with the complex frequency 
Po= -Po + jwo. It results in ambiguity of using the traditional frequency response of filters FIR 


filters with asymmetric form of the impulse response for analysis aperiodic signals. 


|K(-B + jte 


wi 
Rh 
WS 


Figure 9. Amplitude-frequency response of the filter 


In the case of a need along with estimating the speed and accuracy of signal processing to 
appraise the history of transient processes in a filter, for instance, to control oscillation of 
transient process in a filter, it is eligible to conduct the analysis using amplitude-frequency 
response based on the filter transfer function, which depends on time. 


It is necessary for analysis to depress a damped oscillatory component with the complex 
frequency py=—f y+ jwg to fix the imaginary part, due to wy=const. The plot, proportional to 
the product | K,(-B + jwg t) le™, is given in the Fig.10. In the case of complex frequency 
p=—f + ja the plot will be equal to the envelope (curves 1 and 2) of the reaction of the second 
filter (curve 3) to the examined input impact at B)=50 sec". 


The graph presented describes graphically the fact, that the second filter does not ensure the 
suppressing of an input signal component with the complex frequency pp=—fy + jw by the 


moment of free components completion in the filter at the most specified values of Bo. 


3.3. Digital FIR filters 


As for IIR filters (Table 1 and 4), input-output dependencies for digital (discrete) FIR filter may 
be obtained by discretization of expressions for analog FIR filter and transition from the 
Laplace transform to Z transform [13]. 


In case of using the mathematical software Matlab, Mathcad etc. for digital FIR filter analysis 
it is enough to get the data about complex amplitudes and input signal frequencies X (X at 
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finite signals), p (or z=e”") and a filter G, G, q (or z=e1"), as well as a set of parameters, 
which define the duration of filter impulse function components and the beginning and 
duration of input signal components. 


In some cases only mathematical description of analog filter-prototypeis specified. In case of 
FIR filters, different methods of transition from analog filter-prototype description to digital 
filter description, for instance, method of differential equation discretization, method of 
invariant impulseresponses, bilinear transformation are applied [14]. The mentioned methods 
expand also for the case of FIR filters in the author’s paper [13]. 


The transfer function of analog filter-prototype with finite impulse response according to the 
Table 6 can be described in the following way 


K(p) =Re a : | "| i er] (2) 
P—Pin M P—Pin M 


The expression for a digital filter transfer function 


Ke=ne{er| “9 ened] | @) 
M M 


a - coefficients; z,,,- m-th pole of system function, N „T - duration of the m-th impulse 


where k 


m’ 


function component, T- discrete sampling step. 


All the mentioned constants, except N „, depend on the transition method being applied. The 


m’ 
values of the given coefficients for the three transition methods mentioned above are repre- 
sented in the Table 9 


Ne Method Meaning 
k =T,d=0, 


m 
1. Method of differential equation discretization 
Zim=1/(1-P1m): Pi = 12pm) / T 


k,=T,a=0, 
2. Method of invariant impulse responses put 
Zym=e ” 1 P2,,=Pm 


km=T /(2-PpT) 2=1, 23m=(2 + PT) (2=PmT 


3. Bilinear transformation 
03, =!0(Z3m) / T 


Table 9. Transition methods 
The additional indexes of the constants in the Table 9, corresponding to order numbers of 


transition methods, are given for the constants, which do not coincide with the analog filter- 
prototype parameters. 
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4. Performance express-analysis at modulated signal 


4.1. Mathematical description of signals 


The signals as a set of semi-infinite or finite damped oscillatory components were considered 
above. Compound signals of different forms, including compound periodical and quasi- 
periodic signals, nonstationary signals and signals with compound envelopes can be synthe- 
sized on the basis of the collection of components mentioned above [5,6]. The mentioned 
models also make it possible to describe the majority of impulse signals, which are widely 
applicable in radio engineering (radio pulse and video pulse). 


The analysis methods considered above can be applied for that kind of signals. 


However, the more general case is obviously more interesting, when signals with compound 
dependencies of envelopes and signal total phase are applied. Semi-infinite or finite signals 
with compound envelopes, the most frequently used signals in radio engineering, can be 
described by the following model 


x(#) = Re(X; (#)e”") (4) 
or in general case it would be 


x(t) = Re(X(#)' oie (5) 


For the considered signal models, for instance, of the signal (1),their finite duration can be 
specified by the finite duration of a complex amplitude X ,(t). 


Using the model (1), signals with amplitude, phase and frequency modulation which are 
commonly used in radio engineering signals can be described. Among with the similar models 
input signals in sound processing systems, automation devices of power systems at electro- 
mechanical transients process can be described. 


It is known, that using the signal (1), in case when amplitude, frequency or initial phase are 
time functions, it is impossible to define the law of amplitude variation and a filter initial phase 
by values of amplitude-frequency response and phase-frequency response on an input signal 
frequency [21,22]. Thus, the analysis methods considered above cannot be applied directly. 


There are exceptional cases when at some variation laws X ,(t) and p,(t) signals of the type (2) 
can be transformed into signals, described by a set of semi-infinite or finite damped oscillatory 


components [5,6]. 


However, signals (1) and (2) canbe decomposed into components of a set of damped oscillatory 
components using the Prony's method and its modifications [23]. Leaving behind the issue 
concerning decomposition into “real” components by using the mentioned methods it is 
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important to note that even general signal approximation (1) and (2) by Prony's method for 
calculations enables to apply the considered filter analysis methods. 


Another option for solution of the filter analysis at signal type (1) is connected to modification 
of the suggested analysis methods. Modification of the second analysis method is represented 
below. 


4.2. Filter analysis 


Let us consider IIR filter analysis at input signals (1), as well as for special cases, when 
amplitude X (t)=X be i? and phase X (t)=X ne O are corresponding to a modulation 


signal. 


To develop the necessary dependences one can use expressions, obtained for IIR filter at 
injection on its input a set of finite damped oscillatory components. Let us decompose the 
signal into time steps, during which signal time dependent parameters are mostly constant. 
In this case time step At can be even and uneven. 


In case of an even step of signal decomposition one will obtain the following expression to 
determine a filter ouput signal 


N-1 
y(t) = rel X, ert KOL - nA) ePri (=at) NK (prast -(n+ gade (6) 


n=0 


where X „=X (n4t), pa =p (nat), N =t, /At, ty- duration of a finite signal (beginning of the 
signal coincides with zero reading on time). 


As before, only algebraic operations with complex amplitudes, frequencies and values of time 
dependent transfer function on complex frequency of an input signal are applied for the filter 
analysis at the input signals type (1). 


The same approach can be used for digital filters as well —- one should replace continuous time 
t to discrete time kT, and instead of transfer function K (p, t) of an analog filter one should use 
a transfer function K (z, k)in the expression. In this case, if one assumes At to be equal to 
discrete sampling step T, it enables to take into account the errors of analog-to-digital converter 
and finite digit capacity microprocessor in filter analysis. 


The filter analysis on the basis of the expression (3) is approximate and can be considered as 
numerical method. To determine explicit dependencies one need to perform passage to the 
limit At — 0. 


The required dependency of input-output can be defined also by using the convolution integral 
by substitution of the expression for the input signal (1). Performing discretization with the 
passage to the limit, the following input-output dependency for an analog filter-prototype can 
be obtained 
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y(t) =Re [Xie 2)e"'K (p, (2),2) de (7) 


where 


dK(p, (t),t) 


dt e) 


K (p,(t),t) = 


In case of amplitude, phase modulation or their combination, the expression for input-output 
would significantly simplify 


vo sto ee a 


For IIR filter the following expression for a derivative of time dependent transfer function takes 
place 


K (pi) Re - Re( [Gye], ) (10) 


T 


In case of X 1t)= X , the input-output dependency coincides with input-output dependency 
for IIR filter obtained before (item 6 Table 1) 


y(t) =Re(X;K(py-t)e”") (11) 


It follows from the comparison of the input-output dependencies (4) and (5), that in the second 

case complex amplitude of IIR filter output signal on the input injection as a damped oscillatory 

component is determined by multiplication of a signal complex amplitude by the value of time 

dependent transfer function on the input signal frequency Y (t)=X ,K(p,, t), in the first case 
t 

more complicated dependency takes place Y ()=[x (t-7)K (p, t)dt. 


0 


The dependencies enable to perform filter analysis, including performance analysis of signal 
processing, at input signals with a composite form. Solving problems of this kind is relevamt 
not only for radio engineering and communication systems, but also for other industries. 


For the example of considered in the sections 3.2 and 3.3 analysis of frequency filters, which 
are used in PMU, according to a new version of IEEE C37.118.2 standard it is necessary to 
perform testing for the mentioned devices not only at stepwise change of amplitude and initial 
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phase of sinusoidal component of input signal with commercial frequency, but also at PMU 
input signals at electromechanical transient process in a power system. In the mentioned 
regimes of power system operation in the controlled currents and voltage envelopes and total 
phases are time functions. 


For FIR filter analysis at input actions (1) the input - output expression can be obtained on the 
basis of the dependences given in the Table 6. Such-like dependences for IIR filters and FIR 
filters can be obtained based on the dependences given before for the first analysis method. 


5. The application of the analysis methods 


Let us consider possible areas of application for the suggested performance analysis methods 
for signal processing by frequency filters, which are used in intelligent electronic devices of 
power systems, in automation devices, in radio engineering and communication systems, as 
well as in other fields of engineering where digital signal processing is commonly applied. 


The prospectives of power system development in nearest future are related to technology 
Smart Grid implementation and the application of automatic control and regulation systems 
of a new generation. Power system control improvement involves a wide application of fast 
action IED based on synchronized measurement of current and voltage phasors of a funda- 
mental harmonic on the basis of IEEE C37.118-2011 and IEC 61850-90-5 standards. 


Up to date IED should ensure performance signal processing in conditions of an intense 
electromagnetic and electromechanical transients process. Mathematical model of an input 
signal IED in a normal and an accidental regimes of power systems in some cases can be 
represented by a set of semi-infinite or finite damped oscillatory components, in other cases — 
by analogous models, in which complex amplitudes and frequencies of mentioned compo- 
nents are time functions. 


Most of IED power systems should ensure performance of signal processing at any possible 
combination of input signal parameters. The suggested analysis methods enable to solve 
effectively problems of determination of performance specifications for frequency filters, used 
in IED power systems. The examples for the performance analysis of signal processing by 
frequency filters and algorithms of signal processing for IED of power systems based on the 
phasor measurement technology are considered in the sections 3 and 4 of the present chapter. 
The example for IIR filter analysis for general devices of relay protection and automation is 
given in the section 2. 


The analysis methods for analog IIR filters considered in the chapter can be applied also for 
the linear circuit analysis. It is important to note that for absolute majority of microprocessor 
control systems and measuring systems the information sources have analogue nature. In this 
case for analysis of controlled objects equivalent circuits based on linear electric circuits are 
used. The illustrative examples: equivalent circuit of power systems, power plants, electric 
grids and power-supply systems. Application of the suggested analysis methods enables a 
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consistent approach for the regime analysis of the controlled object operation, analysis of 
analog filters-prototype and digital filters [5,6]. 


The suggested analysis methods may be applied for performance analysis of signal processing 
by frequency filters in up-to-date measuring devices, automation devices, radio devices, in 
communications systems, sound processing systems and other devices, where digital signal 
processing is commonly used. 


Advantages and particularities of the suggested analysis methods are related to uniform 
analysis methods for analog filter-prototypes (IIR filters and FIR filters), as well as for digital 
filters. 


The majority of pulse signals (radio and video pulses), which are commonly used in radio 
engineering, can be described by a set of finite damped oscillatory components. For perform- 
ance analysis of pulse signal processing by IIR filters and FIR filters the analysis methods 
considered in sections 2 and 3 of the chapter can be applied. 


The author suggests to use the methods, considered in the section 4 of the present chapter for 
performance analysis of modulated signal processing by filters. 


The synthesis methods mentioned above can be also effectively applied for typical signal 
filtering problems, including problems of a useful signal extraction against the white noise. 


In this case, the white noise realizations can be described as a set of time-shifted fast damping 
exponents of different digits. The initial values and appearance time of the mentioned 
exponential components are random variables, which variation law ensures the white noise 
to have specified spectral characteristics. 


6. Performance analysis of signal processing as a step of filter synthesis 


Guaranteeing the necessary quality of signal processing is utterly important at frequency filter 
synthesis. The application of the considered above approaches for the signal type (1) and the 
filter impulse characteristics (2) enables to reject the traditional approach, related to formula- 
tion the requirements to a filter amplitude-frequency response in different fields (band pass, 
stop band, transition region). In the case of filter synthesis, it is enough to lay down the 
requirements to the filter frequency characteristics on the basis of Laplace transform on 
complex frequencies of an input signal with the allowance for their change. Thus, according 
to the approach described above, the signal processing performance analysis is one of the steps 
of filter synthesis. 


This approach enables to formalize the task of filter synthesis and to gain optimal solutions in 
combination with methods of multicriterion optimization with limitations[5]. Using methods 
of multicriterion optimization, by limitations values of filter frequency responses are ment and 
in some cases values of input signal spectrum based on the Laplace transform spectral 
representations on complex frequencies of impulse function of a filter and input signal [5,20]. 


Direct Methods for Frequency Filter Performance Analysis 
http://dx.doi.org/10.5772/52192 


Synthesized in that way filters ensure the specified performance specifications for signal 
processing at any possible variation of input signal parameters. 


Among with frequency filter synthesis this approach gives an opportunity to perform time 
window synthesis for hort-time Fourier transform, as well as synthesis of father and mother 
wavelets for cases when an input signal can be represented by a set of semi-infinite or finite 
damped oscillatory components [5,20]. 


7. Conclusion 


The effective methods of performance analysis for signal processing based on the Laplace 
transform spectral representations and the uniformity of mathematical models of input signals 
as well as the filter impulse functions as a sets of continuous/discrete semi-infinite or finite 
damped oscillatory components were developed for a direct determination of IIR filters and 
FIR filters performance specification. input signals as well as the filter impulse functions as a 
set of continuous/discrete semi-infinite or finite damped oscillatory components were 
developed. Simple semi-infinite harmonic, aperiodic signals, compound signals and impulse 
characteristics of any form can be synthesized on the basis of components set mentioned above, 
including signal composite envelopes, as well as pulse signals (radio and video pulses). The 
uniformity of the mathematical description of the signals and filters enables, on one hand, 
allows to employ both a consistent and a compact form of their characterization in the 
configuration of a set of complex amplitudes, complex frequencies and time parameters. On 
the other hand, it simplifies significantly performance analysis for signal processing by analog 
or digital filters at any possible useful signal and disturbance parameters variation by reducing 
significantly the amount of calculations. 


The analysis methods can be used in case of mathematical models as well - where complex 
amplitudes and/or complex frequencies are time functions. 


To simplify the task of the analysis, two methods are suggested for performance express- 
analysis of signal processing by frequency filters using filter frequency responses based on 
Laplace transform: frequency and frequency-time analysis methods. 


The application of the suggested methods for performance analysis of signal processing as one 
of the steps of filters synthesis enables to automatize filter design for filters with low sensitivity 
to a signal parameter change within the specified range. 
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1. Introduction 


Frequency transformation is one of the well-known techniques for design of analog and 
digital filters [1, 2]. This technique is based on variable substitution in a transfer function 
and allows us to easily convert a given prototype low-pass filter into any kind of frequency 
selective filter such as low-pass filters of different cutoff frequencies, high-pass filters, 
band-pass filters, and band-stop filters. It is also well-known that the transformed filters 
retain some properties of the prototype filter such as the stability and the shape of the 
magnitude response. For example, if a prototype filter is stable and has the Butterworth 
magnitude response, any filter given by the frequency transformation is also stable and 
of the Butterworth characteristic. Due to this useful fact, the frequency transformation is 
suitable not only to the filter design but also to the real-time tuning of cutoff frequencies, 
which can be applied to design of variable filters [3] and to adaptive notch filtering [4, 5]. 
Hence the frequency transformation plays important roles in many modern applications of 
signal processing from both the theoretical and practical points of view. 


The purpose of this chapter is to provide further insights into the theory of frequency 
transformation from the viewpoint of internal properties of filters. In many textbooks on 
digital signal processing, the frequency transformation is discussed in terms of only the 
input-output properties, i.e. properties on the transfer function. In other words, few results 
have been reported about the relationship between the frequency transformation and the 
internal properties. As is well-known, the internal properties of filters are closely related to 
the problem of how we should construct a filter structure of a given transfer function, and this 
problem must be carefully considered in order to obtain analog filters of high dynamic range 
and low sensitivity [6-12] or digital filters of high accuracy with respect to finite wordlength 
effects [13-25]. Hence it is worthwhile to investigate the frequency transformation from the 
viewpoint of the internal properties, and to extend the results to some practical applications. 
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In order to discuss the frequency transformation from the viewpoint of the internal 
properties of filters, we make use of the state-space representation. The state-space 
representation is one of the well-known internal descriptions of linear systems and, in 
addition, it provides a powerful tool for synthesis of analog/digital filter structures with 
the aforementioned high-performance. The results from our discussion are twofold. First, 
we reveal many useful properties of frequency transformation in terms of the state-space 
representation. The properties to be presented here are closely related to the following 
three elements of linear state-space systems: the controllability Gramian, the observability 
Gramian, and the second-order modes. These three elements are known to be very 
important in characterization of internal properties of analog/digital filters and synthesis 
of high-performance filter structures. Second, we apply this result to the technique of 
design and synthesis of analog and digital filters with high performance structures. To 
be more specific, we present simple and unified frameworks for design and synthesis of 
analog /digital filters that simultaneously realize the change of frequency characteristics and 
attain the aforementioned high-performance. Furthermore, we extend this result to variable 
filters with high-performance structures. 


The chapter is organized as follows. Section 2 reviews the fundamentals of the state-space 
representation of linear systems, including analog filters and digital filters. Section 3 
introduces the classical theory of frequency transformation. Sections 4 and 5 are the main 
theme of this chapter. In Section 4 we discuss the frequency transformation by using 
the state-space representation and reveal insightful relationships between the frequency 
transformation and the internal properties of filters. In Section 5 we extend this theory and 
present new useful methods for design and synthesis of high-performance analog/digital 
filters. 


2. State-space representation, Gramians and second-order modes 


In this section we introduce state-space representation of linear systems. In addition, 
we introduce the aforementioned three elements on the internal properties—controllability 
Gramian, observability Gramian, and second-order modes—and we address how these 
elements are applied to synthesis of high-performance filter structures. We will present 
these topics for digital filters and analog filters, respectively. 


2.1. State-space representation of digital filters 


Consider the following state-space equations for an N-th order stable 
single-input/single-output linear discrete-time system: 


x(n+1) = Ax(n) + bu(n) 
y(n) = cx(n) +du(n) (1) 


where u(n), y(n) and x(n) € RN*! denote the scalar input, the scalar output and the state 
vector, respectively, and A € RNXN b © RNXLc e RIXN and d € R!*! are constant 
coefficients. Throughout this chapter we assume that the system is stable, controllable 
and observable. If this state-space system represents a digital filter, each entry of x(n) 
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corresponds to each output of delay elements of the filter. Taking the z-transform of (1), 
we have 


N 
x 

ea 
N 

a3 
II 


AX(z) + bU(z) 
cX(z) +du(z) (2) 


ne 
a 
N 
Sar 

II 


from which the transfer function H(z) is described in terms of (A, b, c,d) as 
H(z) =d+c(zIy — A) 1b (3) 


where Iy denotes the N x N identity matrix. 


It is well-known that the transfer function H(z) is invariant under nonsingular 
transformation matrices T € RNXN of the state: if x(n) is transformed into ¥(n) = T x(n), 
then the state-space system (A, b, c,d) is also transformed into the following set (A, b,c, d): 


(A,b,¢,d) = (T1AT,T tb, cT,d). (4) 


It is easy to show that the transfer function of this new set is the same as that of (A, b, c,d). 
Therefore, many structures exist for a digital filter with a given transfer function H(z). This 
nonsingular transformation is called similarity transformation. 


We next introduce the controllability Gramian, the observability Gramian, and the 
second-order modes. For the system (A,b,c,d), the solutions K and W to the following 
Lyapunov equations are called the controllability Gramian and the observability Gramian, 
respectively: 


K = AKA! + bb! 
W = ATWA + cfc. (5) 


The Gramians K and W are symmetric and positive definite, i.e. K = KT > 0 and W = 
wis 0, because the system (A,b,c,d) is assumed to be stable, controllable and observable. 
Then, the eigenvalues of the matrix product KW are all positive. We denote these eigenvalues 
as 02, 05, a „03, and assume that o? > 05 >.> 03. Their positive square roots 6, > 02 > 
-++ > Oy are called the second-order modes of the system. In the literature on control system 
theory, the second-order modes are also called Hankel singular values because 61, 62,--+ , On 
are equal to the nonzero singular values of the Hankel operator of H(z). 


The two Gramians and the similarity transformation x(n) = T~!x(n) are simply related as 
follows: the controllability /observability Gramians (K, W) of the system in (4) are given by 


(K,W) = (TKTT, TTWT). (6) 
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On the other hand, the second-order modes are invariant under similarity transformation 
because of the following relationship 


KW = T! (KW)T. (7) 


Hence it follows that the Gramians depend on realizations of the system, while the 
second-order modes depend only on the transfer function. 


In the literature on synthesis of filter structures [13-25], it is shown that the two Gramians and 
the second-order modes play central roles in analysis and optimization of filter performance 
such as the roundoff noise and the coefficient sensitivity. In other words, given the transfer 
function of a digital filter, we can formulate some cost functions with respect to the 
aforementioned filter performance in terms of the two Gramians (K, W), and a filter structure 
of high performance can be obtained by constructing the two Gramians appropriately in such 
a manner that they optimize or sub-optimize the corresponding cost functions. 


An example of high-performance digital filter structures is the balanced form [15, 16, 18, 23, 
25]. This form consists of the two Gramians given by 


K=W=0 (8) 


where @ is the diagonal matrix consisting of the second-order modes, i.e. 


(3) = diag (61, 62, ERR ,ONn). (9) 


Another example is the minimum roundoff noise structure [13, 14, 16, 17], which consists of 
the two Gramians that satisfy the following relationships 


yy V 
Kj =1 (10) 


where K;; denotes the i-th diagonal entry of K. 


Finally, we address the significance of the second-order modes from two practical aspects. 
First, it is known in the literature that the second-order modes describe the optimal values 
of the aforementioned cost functions. Therefore, it follows that the optimal performance 
is determined by the second-order modes of a given transfer function. Another important 
feature of the second-order modes can be seen in the field of the balanced model reduction 
[26-28], where it is shown that the second-order modes provide the upper bound of the 
approximation error between the reduced-order system and the original system. 
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2.2. State-space representation of analog filters 


An N-th order linear continuous-time system (including analog filter) can be described by 
the following state-space representation 


2 = Ax(t) + bu(t) 
y(t) = cx(t) + du(t) (11) 


where u(t), y(t) and x(t) € #N* are the scalar input, the scalar output and the state vector 
of the system, respectively, and A € RN*N b e RN*1, c € RIN and d € R'*! are constant 
coefficients. The system (A,b,c,d) is assumed to be stable, controllable and observable. If 
this system represents a continuous-time analog filter that comprises N integrators, the state 
vector corresponds to the output signals of these integrators. 


Taking the Laplace transform of (11) leads to 


sX(s) = AX(s) + bU(s) 
Y(s) = cX(s) +dU(s), (12) 


which results in the following transfer function 
H(s) = d + c(sIy — A) ‘b. (13) 


As similar to the discrete-time case, the transfer function is invariant under similarity 
transformation: if x(t) is transformed by a nonsingular matrix T € RN* into T~!x(t), then 
the new state-space system (T-!AT, T~!b,cT,d ) is an equivalent realization to (A, b, c,d) of 
the transfer function H(s). Therefore, many circuit topologies exist for an analog filter with 
a given transfer function H(s). 


The controllability Gramian K and the observability Gramian W of a continuous-time 
state-space system are respectively obtained as the solutions to the following Lyapunov 
equations: 


AK + KA! + bb! = OyxN 
ATW + WA + c'c = Onxn (14) 


where 0); denotes the N x N zero matrix. By the assumption of the stability, controllability 
and. observability of (A,b,c,d), the Gramians K and W are shown to be symmetric and 
positive definite. Then, as in the discrete-time case, the second-order modes 01, 62,--- ,ON 
are obtained as the positive square roots of the eigenvalues of KW. 


The relationship of similarity transformations to the Gramians and the second-order modes 
in the continuous-time case is the same as that in the discrete-time case. The new Gramians 
(K,W) of the transformed continuous-time system given by a similarity transformation T 
are shown to be (T~'KT~', T'WT), and thus the Gramians depend on realizations of the 


system. On the other hand, the second-order modes are invariant because KW = T~!(KW)T 
holds. 


113 


114 Digital Filters and Signal Processing 


As in the discrete-time case, the Gramians and the second-order modes of continuous-time 
systems play important roles in synthesis of filter structures of high performance [6-12]. A 
high-performance structure can be obtained by optimizing or sub-optimizing a prescribed 
cost function in terms of the controllability and observability Gramians. Such a cost function 
can be seen as a measure of the dynamic range and the sensitivity of an analog filter. In 
addition, the optimal values of such cost functions are determined by the second-order 
modes. 


3. Frequency transformation 


3.1. Frequency transformation of digital filters 


Frequency transformation of digital filters can be seen in the work of Oppenheim [29] and 
Constantinides [2]. The work of Oppenheim is applied to finite impulse response (FIR) 
transfer functions, whereas the work of Constantinides is applied to infinite impulse response 
(IIR) transfer functions. In this chapter, the frequency transformation of digital filters is 
restricted to the work of Constantinides. 


Now let H(z) be the transfer function of a given N-th order digital low-pass filter. The 
frequency transformation in the discrete-time case is defined as 


H(F(z)) = H(Z)|,-1.-1/F(z) (15) 


which results in a new composite transfer function H(F(z)). The function 1/F(z) for this 
transformation is defined as an M-th order stable all-pass function of the form 


l 4y-M Gia") 
F(z) G(z) 
M 
G(z) =14+ } az. (16) 
k=1 


The well-known typical frequency transformations make use of the following four types of 
all-pass functions 


1 ze 
Fp z) 1— éz-1 
1 zite 


le o” qi” irl 
Fpp(z) sy. — 2615-1 4 tel, -2 
BP ) 1 TFI + qa 
—2_ 24-1 1-9 
1 Z- — Tm tT 


= a (17) 
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which respectively correspond to the low-pass-low-pass (LP-LP), low-pass-high-pass 
(LP-HP), low-pass-band-pass (LP-BP) and low-pass-band-stop (LP-BS) transformations. The 
parameters ¢ and y determine the cutoff frequencies of the transformed filters. On the block 
diagram of a digital filter, the frequency transformation means that each delay element zo} 
in H(z) is replaced! with an all-pass filter 1/ F(z). 


3.2. Frequency transformation of analog filters 


Let H(s) be the transfer function of a given N-th order analog low-pass filter. The frequency 
transformation of analog filters is defined as the following variable substitution [1] 


H(F(s)) = H(s)|5-1¢-1/F(s)- (18) 


Hence the frequency transformation yields a new composite transfer function H(F(s)) from 
the prototype transfer function H(s). In general, the cutoff frequency of the prototype 
low-pass filter is set to be 1 rad/s. From a circuit point of view, the substitution s~! + 1/F(s) 
means that each integrator 1/s in the prototype filter H(s) is replaced with another system 
with the transfer function 1/F(s). 


The transformation function 1/F(s) is defined as the following Foster reactance function [1] 


T zs) (82 + w?,) (8? + w2) (8s? + w?,) oor ie 
Fs) Ps) L + w51)(s? + wpa) (5? + wa) es 
where G > 0 and 0 < wy, < Wp1 < Wz2 < Wp2 < Wz3 < Wp3 < `++. The Foster reactance 


functions are determined in such a manner that the degree of difference of p(s) and z(s) is 
1, ie. |deg p(s) — degz(s)| = 1. In the case of the well-known typical LP-LP, LP-HP, LP-BP 
and LP-BS transformations, the reactance functions are respectively given by 


1 G 
Fip S = S 

1 

= Gs 

Fup(s 

1 Ei Gs 
Fgp(s) s+ w 

1 G( +w) 20) 
Fps(s s f 


The parameters G, Wp and ws determine the cutoff frequencies of the transformed filters. 


1 To be precise, replacing z~! with another transfer function often yields a delay-free loop. In this case, some extra 


processing such as reformulation of the coefficients of the transformed filter is required after this replacement. 
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It is important to note that the Foster reactance functions are classified into two 
categories—strictly proper reactance functions and improper reactance functions*. In 
the typical frequency transformations of (20), 1/F,p(s) and 1/Fgp(s) correspond to 
strictly proper reactance functions, whereas 1/Fyp(s) and 1/Fgg(s) are improper reactance 
functions. 


4. State-space analysis of frequency transformation 


In this section, we discuss the frequency transformation from the viewpoint of the internal 
properties. In other words, we show many interesting results of the frequency transformation 
in terms of the state-space representation. 


This research has its roots in the work of Mullis and Roberts [30], where they presented a 
simple state-space formulation of frequency transformation for digital filters and they proved 
an important property of the second-order modes—they are invariant under frequency 
transformation. In addition, they provided practical impacts of these results on the design 
and synthesis of high-performance digital filters. 


In this chapter we start with introducing this work, and then we further extend this result and 
present other theoretical results on the relationship between the frequency transformation 
and the state-space representation of discrete-time systems. In addition, we also present 
similar results for continuous-time systems. 


4.1. State-space formulation of frequency transformation for digital filters and 
invariance of second-order modes 


Mullis and Roberts [30] first presented an explicit state-space representation of frequency 
transformation as follows. Let (A,b,c,d) be a state-space representation of a given prototype 
filter H(z). Then, the transfer function H(F(z)) that is given by the frequency transformation 
(15) with an M-th order all-pass function 1/F(z) can be explicitly described by 


H(F(z)) =D+€(zIyn — A) 'B (21) 
with the following coefficients 


A = Iy ®a+[A(In — 6A)~*] @ (By) 

B = [(Iy —6A)~'b] @B 

C = [e(Iy - 8A) 1] 8 7 

D = d + ôc(Iy —5A)~'b (22) 


where («, P, 7,6) is an arbitrary state-space representation of 1/F(z), and ® stands for the 
Kronecker product for matrices. 


? A rational function G(s) = N(s)/D(s) is called strictly proper if degN(s) < degD(s). On the other hand, G(s) is 
called improper if degN(s) > degD (s). Since the Foster reactance functions given by (19) always satisfy | deg p(s) — 
deg z(s)| = 1, there does not exist any reactance function such that deg p(s) = deg z(s). 
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The significance of the description given by (22) lies in the fact that, by using this description, 
we can easily carry out the frequency transformation on a state-space structure as well as a 
transfer function. Also, note that this description does not include any delay-free loop. 


In addition to the above state-space formulation, Mullis and Roberts also described the 
Gramians and the second-order modes of the transformed system (A,B,C,D). The two 
Gramians, which are respectively denoted by K and W, are given as follows: 


K=K®Q 
w=Wweaq! (23) 


where Q is the controllability Gramian of the all-pass system («,B,7,0). From this 
relationship we easily see 


KW = (KW) @Iy (24) 


which means that the matrix product KW have the same eigenvalues as KW with 
multiplicity M. This shows that the second-order modes of transformed filters are the same 
as those of a given prototype filter. Hence the second-order modes of digital filters are 
invariant under frequency transformation. 


The practical benefit of this invariance property is discussed as follows. As stated in Section 2, 
the second-order modes determine the optimal values of cost functions with respect to finite 
wordlength effects. In [30], using the fact that the minimum roundoff noise is characterized 
by the second-order modes, it was proved that the minimum attainable value of the roundoff 
noise of digital filters is independent of the filter characteristics that are controlled by the 
frequency transformation. A similar conclusion can be drawn for the balanced model 
reduction: the upper bound of the approximation error due to the balanced model reduction 
is invariant under frequency transformation. 


Furthermore, in the case of the LP-LP transformation, the work of [30] also presents 
the specific state-space-based frequency transformation that can preserve the optimal 
realizations. This specific transformation is given by 


A = (In + A)(In +A)! 
B = \/1-@(In+GA) ‘1b 


C = /1-@c(In +A)! 


D = d — čc(Iņy + A) Íb. (25) 


By setting the prototype state-space filter (A,b,c,d) to be the optimal realization and 
applying (25), we can obtain arbitrary low-pass filters that have the same optimal realization 
as the prototype filter. 


In the rest of this section, we will provide our results that are derived by further extending 
these results. 
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Figure 1. Gramian-preserving frequency transformation. 


4.2. Gramian-preserving frequency transformation for digital filters 


Here we pay special attention to the controllability and observability Gramians, and we 
provide a new state-space formulation of frequency transformation that can keep these 
Gramians invariant. This new state-space-based frequency transformation is called the 
Gramian-preserving frequency transformation [31] and includes the formulation of (25) as a 
special case. 


Before showing the mathematical formulation of the Gramian-preserving frequency 
transformation, we first discuss how the Gramian-preserving frequency transformation is 
related to design and synthesis of digital filters. Simple examples for design/synthesis of 
low-pass, high-pass, band-pass and band-stop filters are given in Fig. 1. Here, suppose 
that we are given a prototype low-pass filter with the transfer function H(z), as shown 
at the left of this figure. Also, let the controllability/observability Gramians of this 
prototype filter be K and W, respectively. Then, by applying the Gramian-preserving 
frequency transformation to this prototype filter, we can convert this filter into other 
arbitrary low-pass, high-pass, band-pass and band-stop filters that consist of the same 
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controllability /observability Gramians as those of the prototype filter’. Now, recalling that 
high-performance structures can be obtained by appropriate choice of the Gramians, we 
notice that the Gramian-preserving frequency transformation is a very powerful technique 
for simultaneous design and synthesis of high-performance digital fitlers. That is, if we 
prepare the structure of a given prototype low-pass filter as a high-performance one such as 
the balanced form and the minimum roundoff noise form, the Gramian-preserving frequency 
transformation enables us to obtain other types of filters with the same high-performance 
structure. This fact is also true for analog filters, as will be shown later in the next subsection. 


We now present the mathematical formulation of the Gramian-preserving frequency 
transformation. Given a prototype state-space digital filter (A,b,c,d) with the transfer 
function H(z) and an M-th order all-pass function 1/ F(z), the following description provides 
the Gramian-preserving frequency transformation to produce the composite transfer function 
H(F(z)): 


A = # ® In + (BY) @ [Aln — 6A)“1] 

B = B® [(In —6A)~'b] 

Č = 7 8 [c(Iy — 6A)“ }| 

D =d+dc(Iy — A) tb (26) 


where the set (œ, B, y, ô) is a state-space representation of 1/F(z) with the 
controllability /observability Gramians equal to the identity matrix, i.e. 


we" +BB =e e+ 7H = Im. (27) 


This relationship means that the set (@, B, 7,6) is a balanced form. It should be noted that 
such a set always exists if 1/F(z) is stable. 


Now we turn our attention to the mathematical formulation of the Gramians of (A, B, Č, D), 
which are respectively denoted by K and W. They are given in terms of the Gramians of 
the prototype filter as follows: 


K =IM8K 
W = IW (28) 


which means that K and W become block diagonal matrices with M diagonal blocks all 
equal to K and W. Therefore, as stated earlier, K and W respectively become the same 
as K and W with multiplicity M. Hence (26) preserves the Gramians under frequency 
transformation. 


3 In the case of LP-BP and LP-BS transformations, the transformed filters have the same Gramians with multiplicity 
2 as those of the prototype filter. This is because the all-pass functions 1/Fgp(z) and 1/Fgg(z) are second-order 
functions and the order of H(Fgp(z)) and H(Fgg(z) become twice as high as that of H(z). 


119 


120 Digital Filters and Signal Processing 


We next discuss the Gramian-preserving frequency transformation from a realization 
point of view. From (27), we first see that realization of the Gramian-preserving 
frequency transformation requires us to construct the structure of the all-pass filter 1/F(z) 
appropriately such that its state-space representation becomes a balanced form. Although 
formulation of the balanced form is known to be non-unique for a given transfer function, 
we presented a useful technique [31]: given an all-pass transfer function 1/F(z), its 
normalized lattice structure becomes a balanced form, which enables us to realize the 
Gramian-preserving frequency transformation. This is derived from the fact that 1/F(z) 
is all-pass. Now, recall that the frequency transformation of digital filters means that each 
delay element in a prototype filter is replaced with an all-pass filter (and delay-free loops, 
if any, are eliminated after this replacement)*. In view of this, we can conclude that the 
Gramian-preserving frequency transformation is interpreted as the replacement of each delay 
element in the prototype filter with the all-pass filter that has the normalized lattice structure. 
Figure 2 illustrates this scheme. Given a state-space prototype filter as in Fig. 2(a), we carry 
out the aforementioned replacement and we obtain the transformed state-space filter as in 
Fig. 2(b). The all-pass filter that is included in this structure consists of M lattice sections 
@,,--- ,®m, and each section ®; is given as in Fig. 2(c). The variable ¢; for 1 < i < M 
denotes the i-th lattice coefficient for 1/F(z), and @=V/1-@. 

Finally, we provide the mathematical formulation of the Gramian-preserving frequency 


transformation based on the normalized lattice structure. The normalized lattice structure of 
1/F(z) can be given by the following state-space representation: 


-ő -ë -ibi «+. Aes --- Esme -Hbbs + Em oom -e «Em ae 
61 —6162 —C16263 «++ —616263 ` ** GM-36M-2 —01 6263 ' '*M-26M-1 —616263 ** ` ¢m—16M 


wa | 9 & 803 o —G,€3€4---Eum—s€m—2 —Ey8364---Em—2m—1 —Go636a---Em—1em 
0 0 © 0 Emi E 
O83 ---Eu-iém 
616263 ` + * ČM-1M 
Z 263 * °- $m—16M 
p= : 
Em—2ém—1ém 
€m—1SM 
y= (000---0 êm) 
6 = +ĉm. (29) 


Therefore, substitution of (29) into (26) carries out the Gramian-preserving frequency 
transformation. Note that the state-space representation (A,B,C,D) given in this | way 
becomes sparse due to many zero entries in # and ¥. To be precise, the set (A, B,C,D) 


* Note that the mathematical formulation of the Gramian-preserving frequency transformation (26) is derived after 
elimination of delay-free loops. Therefore, (26) does not have the problem of delay-free loops. See [30] for the details. 


Frequency Transformation for Linear State-Space Systems and Its Application to High-Performance Analog/Digital 
Filters 
http://dx.doi.org/10.5772/52197 


Figure 2. Gramian-preserving frequency transformation: (a) prototype state-space filter, (b) transformed state-space filter, and 
(c) a normalized lattice section ¥;. 


has in total (M —1)N(MN — M/2) zero entries. Hence this state-space filter is very suitable 
to implementation. 


4.3. Results for analog filters 


In the case of analog filters, little had been reported about the state-space analysis of 
frequency transformation. On the other hand, our work [32-34] has derived many results 
that are similar to the discrete-time case. Here we will introduce these results. 


We first present a state-space formulation of frequency transformation for analog filters. One 
thing to be noted here is that, as stated in Section 3.2, the frequency transformation functions 
(i.e. Foster reactance functions) are classified into strictly proper functions and improper 
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functions. In this chapter we focus on the case of strictly proper reactance functions, which 
include the LP-LP and the LP-BP transformations. 


Now consider a state-space representation (A, b, c, d) of a given prototype low-pass filter with 
the transfer function H(s). Also, let (A, B,C, D) be a state-space representation of H(F(s)), 
where 1/F(s) denotes a strictly proper Foster reactance function. Then, (A,B,C,D) can be 
given in terms of (A,b,c,d) as follows: 


A = In ®a+A® (By) 


B=b8Qß 
C=c®@y 
D=d (30) 


where the set (a, p, y) shown here is an arbitrary state-space representation of 1/F (s), ie. 


1/F(s) = y(sIm —a)~'B (31) 


and M is the order of 1/F(s), ie. M = degp(s) in (19). Note that the d-term in a state-space 
representation of 1/F(s) becomes zero because the reactance function is strictly proper. 
Therefore, the state-space-based frequency transformation given here is simpler than the 
discrete-time case (22). 


Next we discuss the second-order modes of analog filters under frequency transformation. 
Let (K,W) and (K,W) be the controllability/observability Gramians of (A,b,c,d) and 
(A,B,C,D), respectively. Using (30), we can prove the following property: 


K=KeP |} 
W=WeP (32) 


where P is the positive definite matrix that satisfies the following relationship called the 
lossless positive-real lemma: 


«TP + Pa = O0uxm 
P= 7". (33) 


From (32) we easily see 
KW = (KW) ® Im (34) 


which proves that the second-order modes of analog filters are invariant under frequency 
transformation. 


We now present the Gramian-preserving frequency transformation for analog filters. Let 
(A, B,C,D) be the state-space filter that is given by this transformation. Then, (A,B,C,D) 
is formulated as 
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A =&@Iy + (BY) @A 


B= 8b 
C=7@c 
D=d (35) 


where (i, B, 7) is a state-space representation of 1/F(s) that satisfies P = Im in (33), i.e. 


p=7'. (36) 


For (A, B, Č, D) described as above, the controllability /observability Gramians (K, W) are 
found to be 


K =Iy@K 


W = Iy@w. (37) 


Needless to say, this relationship is the same as in the discrete-time case (28). Hence the 
Gramians of a prototype state-space filter are preserved under this transformation. 


As in the discrete-time case, formulation of (œ, B.X) is known to be non-unique. In [34], 
we presented a closed-form representation of (œ, B,¥) that will be very suitable to circuit 
implementation. In order to derive this representation, we first rewrite the Foster reactance 
function (19) as the following partial fraction 


1 L Gis i Go 


Fo) yeahs 


(38) 


where Gj,---G, and Gp are all real and nonnegative, and L = |M/2], ie. L is the largest 
integer less than or equal to M/2. Note that Go = 0 holds if M is even. Also, note that 
the first term on the right-hand side of (38) vanishes if M = 1. Now we can formulate the 
desired state-space representation of 1/F(s) by using the parameters of (38). The formulation 
depends on the value of M, ie. the order of 1/F(s). For even M, we give the desired 
state-space representation, which is denoted by (@even, Beven Veven), as follows: 


Keven = block diag (Mp1,.Ap2,- + Apr) 
F T 
=at eT 
Beven = (Fi Pa PL) 
z sAN 
Yeven = Beren (39) 
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where Qp; € R?*? and Pi E€ R?*1 for M = 1,2,--- ,L are respectively given by 


0 Wwpi 
Q; = pt 
p! ee 0 ) 


= G 
ca ( n ; (40) 
If M is odd, we give the desired state-space representation (& qq, Boda; Voda) as 


z dd = Ge ce) 
a 0ix2L 01x1 


Boda a (Boren VGo ) i 


_ ~T 
Yoda = Boda: (41) 


Note that the above expression reduces to (oda Bodar Yoda) = (0, V Go, VGo) if M = 1. By 
direct calculation it is easy to prove that the state-space representations (39) and (41) satisfy 
the transfer function 1/F(s) given by (38) for even M and odd M, respectively, and that they 
also satisfy P = Iņ in the lossless positive-real lemma, i.e. 


Taven + Keven = MxM 
Beren = Fere 
ead + Šoda = OmxM 
Boda = Tsaa: (42) 


This result shows that (39) and (41) offer the closed-form expression for the 
Gramian-preserving frequency transformation. 


Finally, we discuss the physical interpretation of the Gramian-preserving frequency 
transformation, which will bring further insight into the circuit theory. As in the discrete-time 
case, we first discuss the Gramian-preserving frequency transformation in terms of the block 
diagram. As illustrated in Fig. 3, the Gramian-preserving frequency transformation for 
analog filters is derived from the model of Fig. 3(b), which is given by replacing the 
integrators in the prototype filter of Fig. 3(a) with an appropriate state-space representation 
(a, B, Y) of the Foster reactance function 1/F(s). Here, we have to consider how the circuit 
topology of the set (a, B, 7) is constructed. In order to answer this, consider again the partial 
fraction of strictly proper Foster reactance functions 1/F(s) as in (38). This expression is 
well-known as the LC driving-point impedance functions corresponding to the first Foster 
canonical form [1], which is realized by the series connection of a capacitor of capacitance 
1/Gp and L parallel combinations of an inductor of inductance G;/ wri and a capacitor of 
capacitance 1/Gj. 


Figure 4(a) shows the circuit representation of 1/F(s), where 1/F(s) is related to V and I as 
1/F(s) = V(s)/I(s). This circuit is easily expressed in state-space form as 
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(b) 


Figure 3. Gramian-preserving frequency transformation for analog filters: (a) prototype state-space filter, and (b) transformed 
state-space filter. 


E j j 
F ~ Ln 2—4) B; 
+yo(sI1 — x0)" Bo (43) 


where the subsystems («;, B;, y;) for 1 < i < L and (a9, Po, Yo) are found to be 


0 G 
Xi = 0 
G; 
bi = (a) 
v= (10) 
“no = 
By = Go 


yo=1 (44) 
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Figure 4. Construction of desired state-space model of 1/F(s) for Gramian-preserving frequency transformation: (a) LC circuit 
representation of 1/ F(s), (b) state-space model of the LC circuit, and (c) desired state-space model. 


with their state vectors X;(s) and Xo(s) defined as 


T 
Xi(s) = (Vi(s) -1(8)) 
Xo(s) = Vo. (45) 


Figure 4(b) shows the state-space model of 1/F(s) described as above. Substituting (44) into 
(33), we obtain the solutions P; and Pp to the lossless positive-lemma for (;, B;,;) and 
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(æo, By, Yo) as follows: 


P; = diag(1/Gi, Gi/w%;) 
Po = 1/Go. (46) 


From (46) we see that the state-space model of Fig. 4(b) does not satisfy P = Im in (33). 
Hence it is necessary to modify the structure of this model such that P; = In and Po = H 
hold. To this end, we consider the following nonsingular matrices 


To = VGo. (47) 


Note that these matrices satisfy T;TT = p and ToT) = Po E Using these matrices, we 
apply the similarity transformation to (44), which results in the new structure («!, Bi, 7) and 


(a, Bor Yo) as 


i Wpi 0 
Y = ViTi = (VG 0) 


0 = Tp ‘By = VGo 
Yo = YoTo = VGo (39) 


and its corresponding model is given by Fig. 4(c). Then, it immediately follows that this 
modified structure satisfies P = Im in (33) and coincides with the desired state-space 
representations (39) and (41). 


The above discussion shows that the desired structure of Fig. 4(c) is obtained by applying the 
similarity transformation based on (47) to the first Foster canonical form for LC impedance 
networks. Here, it turns out that the nonsingular matrices T;’s and To serve as the scaling 
matrices that convert the matrices P;’s and Pp into the identity matrices. Therefore, we 
conclude that our proposed Gramian-preserving frequency transformation is derived from 
a state-space system of which integrators are replaced with 1/F(s), where the structure of 
1/F(s) is constructed as the scaled version of the first Foster canonical form for LC impedance 
networks. It is interesting to note that this construction of Fig. 4(c) is similar to the realization 
of orthonormal ladder filters [7]: the orthonormal ladder filters are obtained by applying the 
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Ly scaling to the structure of singly-terminated LC ladder networks, whereas the structures 
of Fig. 4(c) is obtained by applying another type of scaling, which makes use of the solutions 
to the lossless positive-real lemma, to the Foster canonical form for LC networks. 


Before concluding this section, it should be noted again that the above results apply 
to the case of strictly proper reactance functions that include the LP-LP and the LP-BP 
transformations. For details of the improper reactance functions such as the LP-HP and 
the LP-BS transformations, see [32-34]. 


5. Application to design and synthesis of high-performance filters 


This section applies the results of the previous section to design and synthesis of 
high-performance analog and digital filters. Emphasis is on the tunable filters, and we 
present a simple method to obtain state-space-based tunable filters with high-performance 
structures. 


5.1. High-performance digital filters 


Here we apply the Gramian-preserving frequency transformation to design and synthesis of 
a variable band-pass filter of high-performance structure [35]. The variable band-pass filter to 
be presented here is assumed to have the fixed bandwidth and the tunable center-frequency. 
Such a band-pass filter requires the simplified LP-BP transformation with the following 
all-pass function: 


= 
1 __,-12 CBP (49) 


Fpp(z) 1—¢pp2—* 


where ¢pp = COS Wpp and wep is the desired center-frequency of the passband in the variable 
band-pass filter. The desired state-space representation of (49) in order to carry out the 
Gramian-preserving frequency transformation (i.e. the state-space representation of (49) with 
the normalized lattice structure) is found to be 


p= yi te 
—Cpp 
7 = (0-1) 
6 =0. (50) 


Substituting (50) into (26), we obtain the state-space representation of the variable band-pass 
filter as 
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y1 — čĝpIĪy ČBrA 


da | Caplin oa 


3- 1 fib 
—Cppb 
Č = (0)xn =c) 
D=d (51) 


and we can easily control the center-frequency of this filter by changing the value of gp in 
(51). 


Now we present a design/synthesis example. The prototype filter used here is the 
fourth-order elliptic low-pass filter with the following transfer function: 


_ 0.0101 — 0.0362z~! + 0.0524z-? — 0.0362z~3 + 0.0101z~4 


H 
(2) 1 — 3,7805z-1 + 5.4142z 2 — 3.4553z 3 + 0.8310z 4 


(52) 


The peak-to-peak ripple, the minimum stopband attenuation and the passband-edge 
frequency of this filter are 0.5 dB, 40 dB and 0.057 rad, respectively. We choose the state-space 
representation (A,b,c,d) of this prototype filter as follows: 


0.9838 —0.1007 —0.0165 —0.0171 
0.1007 0.9582 —0.1029 —0.0273 
—0.0165 0.1029 0.9336 —0.1015 
0.0171 —0.0273 0.1015 0.9139 
b = (0.1490 —0.1953 0.1669 —0.0995)* 
c = (0.1490 0.1953 0.1669 0.0995 ) 
d = 0.0101. (53) 


A= 


The controllability /observability Gramians of this realization are calculated as 
K = W = diag(0.8850, 0.6124, 0.2761, 0.0817), (54) 


which shows that this realization is the balanced realization. 


Applying (51) to (53) yields the eighth-order variable band-pass filter. It can be easily 
checked that, for any gp, the Gramians of this band-pass filter become the same as (54) 
with multiplicity 2, i.e. 


K = W = diag(0.8850, 0.6124, 0.2761, 0.0817, 0.8850, 0.6124, 0.2761, 0.0817). (55) 
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Therefore, the variable band-pass filter keeps the balanced form regardless of the location of 
the center-frequency. 


Figures 5(a), (b), (c) and (d) show the magnitude responses of our proposed variable filter 
for sp = —0.8, —0.4, 0.5 and 0.9, respectively. For comparison purpose, the magnitude 
responses in the case of the cascaded direct form are also shown here, and all the coefficients 
of these two variable filters are quantized to 10 fractional bits. From Figs. 5(a), (b), (c) 
and (d) we know that our proposed variable filter shows very good agreement with the 
ideal magnitude responses for all ¢pp. This result confirms that, our proposed variable 
filter exhibits high accuracy for all tunable characteristics by constructing the state-space 
representation of the prototype filter appropriately with respect to the Gramians. On the 
other hand, the magnitude responses of the cascaded direct form are degraded in all cases 
and the degradation is extremely large for ¢gp = 0.9. As is well-known, direct form digital 
filters are very sensitive to quantization effects. In addition, since variable digital filters 
with direct form do not take into account the controllability /observability Gramians, the 
performance of the direct form with respect to quantization effects highly depends on the 
frequency characteristics. These facts show the utility of our proposed method. 
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Figure 5. Magnitude responses of the eighth-order variable band-pass digital filters: (a) Responses for sp = —0.8. (b) 


Responses for gp = —0.4. (c) Responses for gp = 0.5. (d) Responses for gp = 0.9. 
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5.2. High-performance analog filters 


Here we will design and synthesize a variable analog band-pass filter by using the 
Gramian-preserving frequency transformation. In the LP-BP transformation, we use the 
second-order Foster reactance function 1/Fgp(s) as in (20). Therefore we apply (39) to 
(35), which results in the following state-space formulation of the desired variable analog 
band-pass filter: 


A- ee Wpil 2 
—WpiIn ONxN 
B 
0Nx1 
Č = (V/Ge Oxy ) 
De=d. (56) 
As a design/synthesis example, here we use the following prototype low-pass filter 
1 

H(s) = (57) 


s + 2s2+2s+1° 


This transfer function is the third-order Butterworth low-pass filter with a cutoff frequency 
of 1 rad/s. We give the state-space representation of this prototype filter as the following 
orthonormal ladder structure [7]: 


c=(c, 00) 
d=0 (58) 
with 
(a1, a2, 03, b3, c1) = (0.7071, 1.2247, 2.0000, 0.7979, 1.4472). (59) 


From (58) and (59), the controllability /observability Gramians of this filter are found to be 


K=; 


16.4493 9.3052 3.7988 
Ww 9.3052 9.8696 5.3723 
3.7988 5.3723 3.2899 


(60) 
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Figure 6. Prototype filter based on transconductance-capacitor integrators. 


As seen above, the controllability Gramian of the orthonormal ladder structure becomes the 
identity matrix. This property brings the high-performance with respect to the dynamic 
range and the sensitivity. Figure 6 illustrates the block diagram of this filter structure based 
on transconductance-capacitor integrators, where the normalized capacitance distribution is 
given by 


(Cp1,Cp2, Cpa) = Cp (0.3091, 0.3957, 0.2952) (61) 


and Cp is the unit-less value of the total capacitance when expressed in F. The specification 
of (61) is determined according to the following rule [10]: 


c= y Ni Wiikii 
f-= S 
Lj y 1j@ iki 
ni = }_ \aijl- (62) 


J 


As is seen from (58) and Fig. 6, the structure of this prototype filter is very sparse and suitable 
for circuit implementation. This is another benefit of the orthonormal ladder structure. 


Applying (56) to this prototype filter, we finally obtain the state-space representation of the 
variable band-pass filter, and its corresponding circuit realization is given by Fig. 7. It can be 
easily shown that the controllability /observability Gramians (£, W) of this band-pass filter 
become 


K = block diag(K, K) = Ig 


W = block diag(W, W) (63) 
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Figure 7. Band-pass filter given by Gramian-preserving frequency transformation. 


for arbitrary values of G and Wp}. It follows from this result that the Gramian-preserving 
frequency transformation easily produces the band-pass filter with the orthonormal ladder 
structure for arbitrary center frequency and bandwidth. Therefore, by controlling the 
parameters of G and wpı, we can realize tunable band-pass filters with the orthonormal 
ladder structure. 


The high-performance of this band-pass filter can be demonstrated by not only calculation of 
the Gramians, but also numerical evaluation of the dynamic range. For details, see [34] and 
the references therein. 


6. Conclusion 


In this chapter we have introduced insightful and useful results on the classical frequency 
transformation of analog filters and digital filters. While most of the known results on 
the frequency transformation are described in terms of the transfer functions, the results 
given in this chapter are based on the state-space representation, which have revealed many 
useful properties with respect to the performance of filters that is dominated by the internal 
properties as well as the input-output relationship. In particular, the Gramian-preserving 
frequency transformation is very attractive to design and synthesis of high-performance 
filters. Using this new frequency transformation, we have presented variable analog/digital 
filters that retain high-performance regardless of the change of the frequency characteristics. 


In addition to the aforementioned work, some other results on the frequency transformation 
have been reported in the literature. One of them is the state-space formulation of 
2-D frequency transformation [36], which presents an explicit state-space-based frequency 
transformation for 2-D digital filters. Also, Yan et al. [37, 38] extended this work to 
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formulations of more general 2-D frequency transformation. Moreover, in [39] we have 
revealed the invariance property of the second-order modes of 2-D separable denominator 
digital filters under frequency transformation. Proof of this invariance property in the case 
of 2-D non-separable denominator digital filters is still an open problem. Derivation of the 
Gramian-preserving frequency transformation in the 2-D case is also an open problem. 


Another interesting topic is the transformations based on “lossy” functions. In both the 
cases of analog frequency transformation and digital frequency transformation, the required 
transformation functions have the lossless property. On the other hand, it is theoretically 
possible to use lossy functions for transformation. Motivated by this, in [33, 40] we presented 
the state-space analysis of lossy transformations and revealed that the second-order modes 
are decreased under such transformations. Development of a practical application of this 
property is a future work. 
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1. Introduction 


Filter banks decompose signals into multiple sub-bands to perform various processes and 
reconstruct the original signals [1]-[8]. It has been demonstrated that sub-band processing 
with filter banks improves the performance of numerous image processing applications 
such as image recognition, watermarking, image coding,and so on [9]-[14]. 


For example, image recognition accuracy is improved with sub-band processing. A water- 
mark is typically embedded into a middle frequency band in order to make the scheme not 
only robust to compression but also secure against attacks on the watermark. Thus, it is 
highly desirable that analysis filters have excellent frequency characteristics to appropriately 
decompose signals in such applications. 


In sub-band coding, on the other hand, we usually focus more on various visual issues 
proper to compression such as checkerboard artifacts and blocking effects, rather than the 
frequency characteristics of the analysis filters. Coding gain is also one of the significant is- 
sues in image coding. Therefore, various constraints are imposed on the filters to deal with 
such problems in the design. Accordingly, the design is usually performed with non-linear 
optimization, which increases its design complexity. 


Multi-resolution processing, which is an extension of sub-band processing, is also in de- 
mand in a number of applications. Multi-resolution schemes such as wavelet transform not 
only decompose the frequencies of signals but also yield various lower resolution versions 
of the original signals by gradually reducing the sampling rate with filter banks. 


Meanwhile, image scaling has become highly required processing in the context of the di- 
versification of the means of displaying images [15]-[17]. Image scaling changes the resolu- 
tion of images more flexibly than multi-resolution processing, although it does not 
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decompose the frequency components of signals. Typically it consists of up/down samplers 
and a filter which suppresses the imaging or aliasing components caused by the samplers. 


Under the circumstances, we discuss complex processes where sub-band processing and 
scaling are carried out sequentially in this chapter. For instance, when watermarked images 
are scaled, or scaling is applied to decoded images, a filter bank and a scaling structure are 
designed separately and implemented sequentially. However, they have the structural simi- 
larity in that they involve samplers and filters. This implies that one can integrate them in 
order to implement them with lower implementation cost. 


Therefore, this chapter provides a profound discussion on a filter bank structure to yield an 
effective scheme in computation in such complex processing. We introduce a simple filter 
bank structure for computational simplicity which directly synthesizes the analysis decom- 
posed signals to produce a scaled signal, not to reconstruct the original signal as in tradition- 
al perfect reconstruction filter banks. 


First, we discuss the frequency decomposition characteristics of the filter bank to show how 
to obtain arbitrarily scaled signals. We briefly clarified the band-width of the synthesis fil- 
ters to obtain the scaled signal from the analysis part of the filter bank in [18]. However, the 
discussion is not sufficient since the behavior of the aliasing components, which can signifi- 
cantly affect the filter bank performance, is not clarified. This chapter discusses the issue so 
that the feature of the filter bank will be manifested. It is shown that the aliasing problem is 
similar to that of perfect reconstruction filter banks. 


Next, theoretical conditions for addressing the above aliasing problem are given through the 
input-output relation of the filter bank. We also discussed the strategy to make the filter 
bank equivalent to the direct scaling structure by deriving such theoretical conditions in 
[18]. However, we observed the number of conditions is proportional to that of aliasing 
components, which can lead to time-consuming design or complicated optimization. More- 
over, we also know the conditions are usually not satisfied when all the filters have linear 
phase, so that the derived filter banks do not have excellent performance. 


In this chapter, we discuss the conditions from a viewpoint of the aliasing components to 
verify that they are the same regardless of the aliasing components, so that the number of 
equations to solve is significantly reduced. A design procedure with the conditions is also 
discussed. We mention that the procedure is applicable to various image processing men- 
tioned above. In addition, it is demonstrated through simulation results that scaled images' 
quality is comparable to that of the sequential structure even though not all the filters have 
linear phase characteristics. Finally, we discuss potential issues and advantages in making 
use of the scheme as well as traditional ones in practical image processing. 


2. Image scaling 


Image-scaling which changes the resolution of images has become highly required process- 
ing in the context of the diversification of the means of displaying images. In such context, 
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various image processing such as image coding, watermarking and recognition is also ap- 
plied before image scaling, in which a filter bank is often accompanied by a scaling structure 
sequentially in practice since filter banks are used in many image processing applications 
with their sub-band decomposition performance. 


Typically, the direct scaling structure consists of up/down samplers and a filter which sup- 
presses the imaging or aliasing components caused by the samplers. Similarly, the filter 
bank structure also involves the samplers and filters. This indicates that one can integrate 
both the structure in order to implement them with lower computational complexity than 
that of their sequential structure shown in Figure 1. 


a(z) 


@ 
D 


HHn G N — ee —(N) Gn a0) 


Figure 1. The typical sequential structure. 


For this, we provide a study on a simple filter bank structure depicted in Figure 2 which is a 
simplified version of the structure given in [18]. Please note that the sampling rate at the 
output in both the structures Figure 1 and Figure 2 is the same when R=U and S=D. In 
addition, the number of channels in both the systems is also the same when N =D . Thus, we 
assume this relation in this chapter. 


The aim of this chapter is to demonstrate that the computational complexity of Figure 2 for 
scaling is reduced compared to that of Figure 1, while its scaling performance is comparable 
to that of Figure 1. 


Hss} — .. A 


Figure 2. The introduced filter bank structure. 
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3. A discussion on the computational complexity of the filter bank for 
scaling 


First, we discuss the computation cost of the introduced filter bank drawn in Figure 2 as 
well as the sequential structure shown in Figure 1. 


The sequential structure decomposes the input signal into sub-band frequency components 
first with down-sampling by the factor D . Next, the decomposed signals are synthesized 
after up-sampling by D . Such sub-band processing with the filter bank is followed by the 
direct scaling structure, where further up-sampling by U is carried out. Thus, the sampling 
rate increases with two successive up-sampling processes. After this, the sampling rate de- 
creases with down-sampling by D . 


On the other hand, the introduced filter bank structure also decomposes the input into mul- 
tiple sub-band signals with down-sampling by D first, and then synthesizes the signals with 
up-sampling by U . 


Accordingly, we see that the sequential structure requires two down-sampling and up-sam- 
pling processes, while the introduced filter bank structure is carried out with only one 
down-sampling and up-sampling process. Such a structural difference can cause different 
computation time for scaling signals. 


If we assume that both the systems have the same length of the analysis filters, then the dif- 
ference in length between both the systems will be in their synthesis part. The difference will 
also cause different computation time. We also discuss these concerns with the design and 
simulation in both of the systems in detail. 


4. A discussion on the filter bank structure for scaling 


4.1. The frequency decomposition and synthesis characteristics of the filter bank 


Next, we discuss the performance of the frequency decomposition and synthesis with the fil- 
ter bank. Figure 3(a) shows an original signal X (z) where the bold line represents 7 [rad] on 
the frequency axis. Figure 3(b) shows the frequency decomposition characteristics with the 
analysis low-pass filter H(z) followed by down-sampling by the factor D . We also show 
the counterpart of the high-pass filter H p4(z) and down-samplingin Figure 3(c). The fre- 
quency components colored with dots represent the aliasing components. Please note that 
these decomposition characteristics in the analysis part are the same as those of traditional 
(perfect reconstruction) filter banks. 
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Figure 3. The frequency decomposition and synthesis in the filter bank 
The analysis signals such as Figure 3(b) and (c) are described in the z domain as 
s3 clive alee Fe 
Y (D=pd X Wz UH AW "z (i=0,1, =, D-1) (1) 
r=0 
where 


2m 


W-e iT (2) 


These analysis signals are up-sampled by the factor U and then filtered with the filters G;(z) 
in the synthesis part, which is illustrated with Figure 3(d) and (e) in the case of G,(z) and 
Gp4(z) , respectively. Please notice that the pass band of the filters G,(z) is not the same as- 


that of H,(z) since the factor U and D are not the same either to yield scaled signals. 


These signals in the synthesis part are represented as 
D- w u 
Y,(2-=p> xlwee*)alwee ®)G-03, ~, D-1) (3) 
r=0 


The eventual scaled signal, which is the output of the filter bank, is drawn in the Figure 3(f), 
and also represented as 
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Y= 5% YG) (4) 


In this manner, one can obtain a scaled signal by using the filter bank, and the scaled signal 
is completely reconstructed when the ideal filters are used in the filter bank. However, it is 
not possible to use such filters, as in traditional filter banks, and the aliasing components af- 
fect the scaling performance with the filter bank. 


Therefore, we provide a discussion on the aliasing components in the next, which manifests 
the scaling behavior of the filter bank. 


4.2. The aliasing components of the filter bank 


The output signal of the filter bank is rewritten as 


V,2)= God X (wre? Hwee?) (5) 


r=0 


When U =D , that is, a perfect reconstruction filter bank is used, then the pass-band of the 
synthesis filters is the same as that of the analysis one. In this case, “serious” aliasing caused 
by the “mismatch” of the band of the filters is avoided, and only minor aliasing caused by 
the non-ideal filters exists in the adjacent areas between the aliasing componentsin the fre- 
quency domain. 


On the other hand, when U #D , which is necessary to obtain an arbitrarily scaled signal, the 
pass-band of the synthesis ones is not the same as that of the analysis ones, and the serious 
aliasing occurs. Actually, it can be observed that the pass-band of G,(z) overlaps with the 
aliasing components in the above equation and also in Figure 3. 


However, it can also be verified in the frequency domain that such serious aliasing is essen- 
tial and necessary to obtain a scaled signal, since the aliasing components themselves form 
the scaled signal in part in thiscase. 


In this sense, one can conclude that even though the input signal is scaled, only minor alias- 
ing between the aliasing components caused by the non-ideal filters is taken into account to 
design the filter banks as well as the traditional filter banks. 


We also compare the direct scaling structure in the sequential structure shown in Figure 1 
with the filter bank of Figure 2 in terms of aliasing. In the direct scaling structure, up-sam- 
pling by the factor of U is followed by down-sampling by D . Hence, when U >D , which 
means the input signal is extended, then down-sampling is not critical sampling, that is, 
oversampling. We illustrate this with Figure 4. 


On the other hand, the filter bank structure performs down-sampling by D first, which 
means critical sampling. This implies that the aliasing caused by down-sampling in Figure 1 
is less than that of Figure 2. Thus, it is also concluded that the filter bank structures have 
more aliasing than the direct scaling structures do when the scaling factors are larger than 1. 
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These are possible issues to address when we design the filter banks that serve as scal- 
ing systems. 


(a) The original signal X{z) 


(b) The filtered signal 


(c) The scaled signal Yg (z) 


Figure 4. The frequency characteristics of the direct scaling structure 


5. The theoretical conditions for the improvement of the scaling 
performance 


Typically, traditional perfect reconstruction filter banks are designed to satisfy perfect recon- 
struction conditions completely or approximately so that aliasing will be eliminated. Simi- 
larly, we provide theoretical conditions for the introduced filter banks to reduce aliasing. 


We see, from Figure 3 and 4, that aliasing in Figure 3 is reduced if the eventual signal shown 
in Figure 3(f) is the same as Figure 4(c). In other words, it is desired that the introduced filter 
banks precisely work as the direct scaling structures. Therefore, a design scheme for the fil- 
ter banks that approximately or exactly serve as the direct scaling structures is provided in 
this section. 
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5.1. The input-output relation in the filter banks and the direct scaling structures in the 
frequency domain 


First, we give the relation between the input and output of the filter banks and the direct 
scaling structures. The output signal of the direct scaling structure in the sequential struc- 
ture shown in Figure 1 is written as 


D-1 vU ES 
Y(2=py x(w P)ulw"z 7). (6) 
r=0 
Please note that we usually use a perfect reconstruction filter bank in Figure 1, therefore, it is 


assumed that the output of the filter bank is the same as its input. 


On the other hand, the output signal Y (z) of the filter bank shown in Figure 2 is represent- 
ed as the equation (4). 


If the signal Y ,(z) is equal to the output signal Y,(z) , then 


ofc 


x (were a lwee Pex lw) lwz ee) (r=0,1, =, D-1) (7) 


in which 


xlwerz P)ex(wre®) (8) 


hold in some cases without any manipulation, or holds in the other cases by changing the 
order of r on the left and right side of (8). This depends on the values of U and D . There- 
fore, the conditions (7) are proportional tothe number of aliasing components D -1 , and the 
parameters in the conditions are H(z) , H,(z) and G,(z) . Moreover, it is observed that the 
conditions are mostly difficult to satisfy when all the filters have linear phase characteristics, 
i.e., the symmetry of the coefficients. Therefore, we give a profound discussion on the condi- 
tions to address these issues in the next. 


5.2. A discussion on the conditions 

First, we discuss the equations (8) to verify that they hold. We assume that U >D , but the 
following discussion is straightforwardly extended to the case where U <D . Since the equa- 
tion 


W f W nD+r (9) 


hold, the equations 
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Ur=nD +r 


(10) 
need to hold in order to satisfy the equations (8). n is an arbitrary integer number. 


If U=D +1 , then Ur=rD +r , which leads to (10). Thus, the equations (8) hold in this case. 
When U#D +1 , (10) does not hold. Hence, we replace r on the right side of (10) with 
r ‘=r(U - D) . Therefore, nD +r ‘=Ur + Dln - r) , which also leads to 


Ur=nD+r.. 
If we replace r ‘with 


(11) 


r "=mod(r $ D) 
then we also obtain 


(12) 


Ur=nD+r". 


(13) 
mod(a, b) represents the reminder in the case where a is divided by b .Therefore, the equa- 
tions 


xlw Urz t)exlwr’s 5) 


(14) 
hold. We see that the equations (14) also hold when U =D +1, i.e., (14) are equivalent to (8) 
when U=D+1. 


Hence, the equations 


ulw ?)-5' Hwr'2 Pez) -01 =, D-1) 
i0 


need to hold in order to satisfy the conditions (7). 


(15) 


Next, we verify the equations (15) are the same regardless of r . When r =r=0, the equa- 
tions (15) are rewritten with h; ee and h r partly (every D -th equation) as 


h apsp=oqD+pS0,1u +k + 1 0,q+1)D+p8o,-aU +k + 


we +h gDrp8iu k + 
h ape ps Zh ogo+p+180,1U +k-1 + Ho, D+p+180,0 -DU +k + e + Ay gD+prgi u +k-1 t 
h ape p12= oqd+p+280,1U +k-2 + 4 0,(g+1)D+p+280,1-U+k-2 + + + Ay gps pe28rusk-2t +e 
h ape p13 oqd+p+380,1U +k-3 + 40,(g+1)D+p+380,(1-1)U +k-3 * 


(16) 
th 1,qD+p+351,1U +k-3 Fan 
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where h; ; , g; ; and h ; are the coefficients of the filter H,(z) , G(z) and H(z) , respectively, 
and the order of the rows except the 0-th row on the right side of (16), however, changes 
depending on the factors D and U . 


Here we define the following vector 
v, =|W cr” W 2er" ua W (D-1)cr ] (17) 


in which c is a constant, and the p “ -th row on the right side of (16) is multiplied by the p Ta 
th element W? “ ineachr’ . For example, the p ” -th row on the right side of (16) is multi- 


plied by the coefficient W ” “whenr =1. 


Moreover, the p “ -th row on the right side includes h ipgD? since the coefficient h i p+D has 
u 
the delay z mlp) , where p ` has the relation to p , which is 
p =mod(U(p +qD), D) (18) 


If we denote p which satisfies p =1 as p, , then c=p, , since the first row on the right side, 
which includes h, pq? is multiplied by the coefficient W P: when r =1.In this manner, the 
constant c is determined. 


On the other hand, we also define the following vector 


v,= [w r W QE xs W (D-1r] (19) 


where the p` -th row on the left side of (16) is multiplied by the p” -th element W Prin 
each r . If r=p, whenr =1, then v, =v, since c=p, . In this case, the equations (16) are the 
same as those inr=r =0. In fact, the equation r=p, holds when r =1, since (18) is equiva- 
lent to (12). In other words, the relation between p and p is the same as that between r ` 
and r . Furthermore, the equations v, =v, hold in all the r“ and r under (12). Therefore, we 


see that the equations (16) are the same regardless ofr orr `. 


However, it is observed that equations are usually not satisfied when all the filters have line- 
ar phase characteristics. Accordingly, we do not impose the symmetry constraints on all the 
filters. We also see that the filter in the desired direct scaling structure needs to have some 
delays in order to satisfy the equations. The performance of such filter banks is examined in 
the next section. 
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6. Design and Simulation Results 


The equations derived in the previous section have unknown parameters H,(z), G;(z) , and 
H(z) . The filter H(z) in the direct scaling structure should be known, or designed, in ad- 
vance. Thus, H,(z) and G,(z) are obtained by solving the equations with a pre-designed de- 
sired H(z) so that the filter bank structure will be exactly or approximately equivalent to the 


direct scaling structure. 


The concrete design procedure depends on applications the filter banks are utilized in. For 
example, several constraints should be imposed on the analysis filters H Az) for the filter 
banks to have non DC leakage, high coding gain, etc., when the filter banks are exploited in 
sub-band image coding. 


The analysis filters should have high attenuation in the stop band, or excellent frequency de- 
composition performance, when the filter banks are employed for watermarking, since the 
watermarks are often embedded only into one narrow sub-band that is robust to attacks. 


These tell that the analysis filters typically suffer from severe constraints. In this section, we 
survey the case where the analysis and synthesis filters are separately designed, i.e., only 
G,(z) are determined so that the equations are satisfied with not only a pre-designed H(z) 
but also H,(z) , in order not to lay much burden on H,(z) . On the other hand, we do not 


impose the symmetry constraints on G,(z) to satisfy the equations more precisely. 


With this design policy, we here design filter banks. As an example, two-channel and three- 
channel filter banks are designed and explored. In the two-channel case, the scaling factor is 
3/2, and in the three channel one, it is 2/3. The length of the linear phase analysis filters is 6 
in the two-channel case, and 8 in the three-channel one. The length of the obtained (non-lin- 
ear) synthesis filters is 9, and 6, respectively. The length of the linear phase filter in the direct 
scaling structure is 3, and 2, respectively, and it has third and half band amplitude charac- 
teristics, respectively. This is summarized in Table1. 


Please note that in the introduced filter bank structures, the direct scaling filters are not re- 
quired for scaling since the filter banks perform not only sub-band processing but also scal- 
ing. The direct scaling filters are used only to design the filter banks. The round brackets 
“( y” are used in this sense in Table 1. It is observed that the direct scaling filters have some 
delays so that the conditions will be satisfied. 


The number of The scaling g g , , 
The length of the analysis, synthesis and direct scaling filters 
channels Factors 
2 3/2 6 9 (3) 
3 2/3 8 6 (2) 


Table 1. The length of the designed filters in the introduced filter banks. 
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For comparison, we also design nearly perfect reconstruction filter banks in the two-channel 
and three-channel case, where the length of the analysis filters is 6, and 8, respectively. 
Please note that the sequential structure consists of the perfect reconstruction filter bank and 
the same direct scaling structure (except the delays). This is also shown in Table2.In the se- 
quential structures, the coefficients of the analysis filters partially have a few zero-values to 
satisfy the perfect reconstruction conditions. The sequential structures also require direct 
scaling filters to perform the scaling process. 


The number of The scaling g , , 
The length of the analysis, synthesis and direct scaling filters 
channels Factors 
2 3/2 6 6 3 
3 2/3 8 9 2 


Table 2. The length of the designed filters in the sequential structures 


The elapsed computation time required for signal scaling in both the sequential structure 
and the introduced filter bank is also compared. The computation is carried out on a com- 
puter with a 2.2 GHz processor. In this comparison, we examine the elapsed time with vari- 
ous scaling factors and length of the filters. 


For example, the elapsed time is 0.2030, and 0.2660 in the introduced structure and the se- 
quential one designed above, respectively, when D=2 , U =3 and the input image size is 
256x256. 


In addition, Table 3 shows the elapsed time[s] when D=5 , U =4, 6, 9, 11 , the length of the 
analysis filters in both systems is 10, and 30, the length of the filter in the direct scaling struc- 
ture is equal to the factor U , and the input image size is 256 x256 . 


The elapsed time[s] The length of the synthesis filters 


The factor U 


Proposed Sequential Proposed Sequential 
4 0.375, 0.406(0.281) 0.437, 0.500(0.359) 8, 24 10, 30 
6 0.390, 0.453(0.297) 0.453, 0.531(0.390) 12, 36 10, 30 
9 0.437, 0.578(0.375) 0.484, 0.563(0.421) 18,54 10, 30 
11 0.453, 0.641(0.391) 0.500, 0.593(0.485) 22, 66 10, 30 


Table 3. Comparison of the elapsed time[s] between the proposed and sequential schemes 


It is observed that the elapsed time is proportional to the factor U and also the length of the 
filters, and that the elapsed time of the introduced filter banks is more than that of the se- 
quential structures when the synthesis filters are long with U = 9, 11. However, this draw- 
back is improved if each branch of the filter banks is completely carried out in parallel, i.e., 
the computation time of the filter banks is the same as that of each branch in the filter banks. 
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We also explored the elapsed time of both systems in this case, which is also shown in the 
round brackets “(- )”, where the length of the analysis filters is also 30, in Table 3. 


We also explore the performance of image-scaling in the two-channel and three-channel cas- 
es. Several well-known test images ( 256 x 256 ) are used as the input to the filter banks. Table 
4 shows the PSNR[dB] of the output of the introduced filter banks against that of the se- 
quential structures. It is observed that the obtained scaled images with the introduced filter 
banks are comparable to that of the sequential structures. However, it is not certain whether 
the proposed approach can yield the optimal solution for image-scaling. A detailed discus- 
sion on this issue is future work. 


The number of 


LENA BARBARA BOAT Building Lighthouse 
channels 
2 147.60 148.88 145.40 146.64 146.56 
3 122.80 123.96 121.02 122.12 121.82 


Table 4. The PSNR[GB] of the introduced filter banks against the sequential structures. 


The introduced filter banks as well as the sequential structures can be applicable to practical 
image processing applications. In this simulation, we examine the performance of both the 
systems in watermarking, where it is assumed that the watermark is embedded into a sub- 
band of the original image and then the watermarked image is rescaled, and that the water- 
mark is extracted from the rescaled image subsequently. We illustrate both the systems in 
watermarking with Figure 5 and Figure 6 in which the scaling factor is 3/2. 


Mr Om OnE Wa We 
at FORLEO AO, 


TH Oe Ja 
Watermark W(z) 


Figure 5. The sequential structure employed for watermarking. 


Scaled signal Yy(z) 


Xo) 
ETARE i oH T 
) 


Watermark W(z Scaled signal ¥,(z) 


Figure 6. The introduced filter bank structure for watermarking. 
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Please notice that the subsequent extracting stage in both the systems requires down-sam- 
pling and also the use of a scaled version of the original image, in order to extract the water- 
mark embedded in a sub-band of the original image in this simulation. The only difference 
between both of the systems is that the sub-band signals are synthesized and then rescaled 
sequentially in Figure 5 while they are rescaled in each sub-band and synthesized in Figure 6. 
This also holds for other image processing applications such as image coding, recognition, etc. 


To demonstrate this, we use several original test images ( 256x256 ) and bi-level watermark 
images ( 128x128 ) shown in Figure 7 and Figure 8 in both the structures, and the water- 
mark is embedded into the high frequency band of the original image because it is known 
that the low frequency one is visually susceptible to the watermarking process. 


the regior 


Figure 8. The bi-level image used as Watermark2. 


Table 5 represents the PSNR[dB] of the extracted watermark against the original watermark 
in both the systemsin which the original image “LENA” is used. It is observed that the ob- 
tained watermarks with the introduced filter banks are comparable to those of the sequen- 


A Study on a Filter Bank Structure With Rational Scaling Factors and Its Applications 153 
http://dx.doi.org/10.5772/ 52391 


tial structures, while the computation time of the proposed filter bank is less than that of the 
sequential structures as shown above. 


Watermark1 Watermark2 


Proposed structure Sequential structure Proposed structure Sequential structure 


34.38 34.56 34.66 34.88 


Table 5. The PSNR[dB] of the extracted watermark against the original watermark. 


We, however, see that the water marks are not extracted with very high PSNR in both the 
systems, which is because re-sampling or filtering is carried out in the image-scaling and 
watermark-extracting stage. 


Typically, watermarks can also be deteriorated in quality or eliminated by attacks such as 
filtering, scaling, etc. Thus, they are more sensitive to such attacks than usual watermarking 
(without scaling) is. A detailed discussion is also the future work. 


7. Conclusion 


This chapter has discussed a filter bank structure which directly synthesizes the analysis de- 
composed signals to produce a scaled signal, which leads to computational simplicity in 
complex processing such as sub-band processing followed by scaling. 


First, we have discussed the frequency decomposition characteristics of the filter bank to 
show how to obtain arbitrarily scaled signals, where the behavior of the aliasing compo- 
nents has also been manifested and addressed, since it can significantly affect the filter bank 
performance as in traditional perfect reconstruction filter banks. 


Next, theoretical conditions for designing the filter bank that is equivalent to the direct 
scaling structure have been given with the input-output relation of the filter bank, where 
it has been mentioned that the number of conditions is proportional to that of aliasing 
components, and that the conditions are mostly not satisfied when all the filters have linear 
phase. Therefore, we have discussed the conditions, so that they are the same regardless 
of the aliasing components, which means that the number of equations to solve is signifi- 
cantly reduced. 


In addition, it has been demonstrated through simulation results that the computation time 
required for signal scaling with the proposed scheme is usually less than that of the sequen- 
tial structures, and that the quality of scaled images is comparable to that of the sequential 
structures even though not all the filters have linear phase characteristics. 


Finally, we have discussed potential issues and advantages in employing the proposed 
scheme as well as traditional ones in practical image processing such as watermarking. 
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The findings in this research are that the derived solution by using the proposed approach 
yields comparable scaling and watermarking performance to that of the usual sequential 
structures that consist of filter banks and also the direct scaling processes, while the compu- 
tation time required for scaling is less than that of the sequential scheme and the design pro- 
cedure is considerabley simplified. 
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1. Introduction 


Accuracy in detection, robustness in performance and the ability to perform accurately and 
robustly are all integral considerations in image related research. However, since applications 
of such research often have a real-time constraint, speed of computation and its corollary - 
time-saving- have also become increasingly important in the research agenda. 


The importance of geometric moment (GM) invariants first received research attention when 
Hu [1] introduced them in his study. As they capture global features have been widely used 
in applications such as object classification, image and shape analysis and edge detection [2-8]. 
However, since GMs are non-orthogonal, a set of continuous orthogonal moments including 
Zernike and Legendre moments was introduced by Teague [9]. Their desirable qualities of 
orthogonality, speed of computation and robustness have found a wider range of applications 
such as character recognition, two-dimensional (2D) direction-of-arrival estimation, and 
trademark segmentation and retrieval [10-13]. However, these continuous moments pose 
certain problems in computation: they require coordinate transformation and suitable 
approximation of the continuous moment integrals. The transformation and approximation 
they require create additional opportunities for the occurrence of error in the computation of 
the feature descriptors. This recognition of the error potential in continuous orthogonal 
moments led to a new set of discrete orthogonal moments such as Tchebichef, Krawtchouk 
and Hahn moments as these do not require coordinate transformation and the discretization 
step in the moment computation [14-18]. 


The computation of both continuous and discrete orthogonal moments can be realized directly 
or via the GMs. The complexity in deriving higher order GMs using a direct calculation 
approach raises a significant challenge for real-time applications. A number of studies address 
this issue using different approaches. Hatamian [19] proposed a novel approach that uses 2D 
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all-pole digital filters with separable impulse responses to implement the first 16 GMs. Wong 
and Siu [20] moved the delay element in the basic filter structure as proposed in [19] from the 
feed-forward to the feedback path. Their method considered moment computation of up to 
the third order. Using a similar structure of cascaded digital filters, Kotoulas and Andreadis 
showed orthogonal moments can be generated from their outputs [21]. To reduce the chip area 
size, they introduced an overflow counter in each of the basic filter structures [22]. However, 
the plurality of large bit-width adders in the digital filter structure presents a major challenge 
to chip synthesis, placement and the routing process. Taking a different tack, Al-Rawi [23] 
generalized the relationship between the moments and the digital filter outputs using a 
recurrence formula. 


Two facts demand our attention at this point. One, in all the aforementioned literature, the 
basic concept involved in the computation of moments using the digital filter structure has 
remained unchanged over the past three decades [19]-[20] even though this digital filter 
structure was formulated when only low orders of moments were used. Two, many recent 
works have involved increasingly higher moment orders. They include, among others, 
invariant image watermarking (30 orders) [24], moving object reconstruction (55 orders) [25], 
and hand shape verification (60 orders) [26]. 


Two models have been proposed to deal with higher order moments. The usage of cascaded 
all-pole digital filters in generating higher order GMs for the formulation of orthogonal 
moments has been successfully explored by Kotoulas and Andreadis. In their work [21]-[22], 
the 40" and 70" high order Zernike and Tchebichef moments, respectively were obtained from 
the digital filter outputs and their transform coefficients. Their digital filter was based on the 
feedforward model, and for a N x N image, the digital filter output values for row filtering 
are sampled at N +2, N +3, N +4, ..., N+p+2, where p is the maximum order. The 
algorithm they propose, however, is undermined by a computational problem. At these time 
instances, the digital filter output values are much larger when compared to the earlier time 
instances. This is because the digital filter operates as an accumulator: their output values 
increase as the number of digital filters directly related to the order increases. The sample of 
digital filter output values obtained from the row filtering for each row are then used as inputs 
to the respective digital filters arranged in the column filtering. This further increases the size 
of the final digital filter outputs. Additionally, since the digital filter used is an approximation 
except for the first two orders, coefficients are then multiplied from second order onwards to 
make them exact. The coefficients, both positive and negative, are determined from the impulse 
response of the digital filter. The model proposed by Wong and Siu [20], though the digital 
filter outputs for all orders are sampled at their respective N , also suffers from a similar 
problem. The current approaches, it is clear, thus suffer from an increase in computational 
complexity arising from the large increase in the digital filter output values as the order 
increases [33]-[34]. 


The method of computation we propose in this work attempts to reduce computational 
complexity — and save time through a reduction in the number of additions — by addressing 
the problems arising from: 1) The increase in digital filter output values as the order of 
moment's increases and 2) the consequent use of positive and negative coefficient multipliers 
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to make them exact. To circumvent the problems arising from these two facts, we are proposing 
the use of only positive coefficient multipliers which then makes it possible to use lower digital 
filter outputs as the order of the moment increases in generation of GMs. As will be discussed 
in greater detail in Section 3, the proposed method is based on the formulation and under- 
standing of the impulse response of the digital filters, the unit step function to be used and 
their relationship with GMs. This formulation makes it possible for the digital filter outputs 
to be evaluated at earlier instancesat N, N-1, N -2, ..., N -p, where the lowest digital 
filter output value, sampled at N - p is for the highest order. Meanwhile, this set of output 
values starts to decrease after p /2 moment orders. 


Recently, another types of discrete orthogonal moments such as Krawtchouk, dual Hahn, 
Racah, Meixner and Hahn moments have been introduced in image analysis community [16]— 
[17]. It was shown that they have better image representation capability than the continuous 
orthogonal moments. 


One main difficulty concerning the use of moments as feature descriptors is their high 
computational complexity. To solve this problem, a number of fast algorithms have been 
reported in the literature [14]-[19]. Most of them concentrated on the fast computation of 
geometric moments and continuous orthogonal moments. This work examines various 
aspects; both theory and applications of image moment implementation using digital filter 
structures. Since these aspects can be discussed rather independently, we devote each chapter 
to the discussion of one particular aspect of moment structures. The following is a summary 
of contents of the sections. 


e Section 2: Orthogonal Moments. Numerous types of orthogonal polynomial, both in 1D 
and 2D, have been described in traditional mathematical literature. In this chapter we 
present a survey of orthogonal moments that are of importance in image analysis. The 
literature on orthogonal moments is very broad, namely in the area of practical applications, 
and our survey has no claim on completeness. We divide orthogonal polynomials and 
orthogonal moments into two basic groups. The polynomials orthogonal on a rectangle 
originate from 1D orthogonal polynomials whose 2D versions were created as products of 
1D polynomials in x and y. The main advantage of the moments orthogonal on a rectangle 
is that they preserve the orthogonality even on the sampled image. They can be made scale- 
invariant but creating rotation invariants from them is very complicated. The polynomials 
orthogonal on a disk are intrinsically 2D functions. They are constructed as products of a radial 
factor (usually a 1D orthogonal polynomial) and angular factor which is usually a kind of 
harmonic function. When implementing these moments, an image must be mapped into a 
disk of orthogonality which creates certain resampling problems. On the other hand, 
moments orthogonal on a disk can easily be used for construction of rotation invariants 
because they change under rotation in a simple way. 


° Section 3: A New Formulation of Geometric Moments from Lower Output Values of 
Digital Filters. In this chapter we propose a new method to accelerate geometric moment’s 
computation using digital filters based on the lower output values. It is shown in this chapter 
a brief reviews of the digital filter methods employed in [19] and [20]. First, a description of 
the proposed method that includes the theoretical formulation of the relationship between 
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the digital filter outputs and GMs is provided. Second, we discuss the computational 
complexity with both an artificial and two real images, and the computational time. Finally, 
this study with some suggestions for future research is concluded. 


° Section 4: A Reduced 2D digital Filter Structure for Fast Computation of Geometric 
Moments. It is shown in this chapter how to design a reduced 2D digital filter grid for fast 
computation of geometric moments. For this design, the 1D and 2D all-pole digital filter 
design procedure using the Z-transform properties is described. It also describes the 
recurrence equations for the desired filter outputs. The work shows the digital filter design 
used in [3] and [4], and shows the implementation of the proposed architecture. Finally, the 
computation results of the proposed method and the method used in [3] is illustrated. 


e Section 5: Conclusions. The presentation is concluded, summarizing the contents of the 
work and discussing possibilities which may be open for the future research. 


2. Moment functions 


Geometric Moments (GMs) and complex moments (CMs) are the simplest among moment 
functions, with the kernel function defined as a product of the pixel power coordinates. These 
type of moments are non-orthogonal and because of this problem, the inverse GMs or CMs 
formulation are not possible. On the other hand, the orthogonal moment functions have been 
used widley for image analysis. The orthogonal moment functions are based on the orthogonal 
polynomials such as Legendre, Zernike, Tchebichef, Krawtchouk and so on. All these moment 
functions play an important role in continuous or discrete domains of polynomials range. 


2.1. Geometric and complex moments 


GMs are defined with the basis kernel {x ” y 1}. The (p +q) * order two-dimensional GMs are 


denoted by m,,, , and can be expressed as 


pq 
Mi” i x Py f(x, y)dxdy (1) 


where f(x, y) is the image intensity function. GMs of low orders have an intuitive meaning, 
Moo is a “mass” of the image (for binary images, my is an area of the object), m_01/m_00 and 


m_10/m_00 define the center of gravity or centroid of the image. Second-order moments mz 
and m, describe the “distribution of mass” of the image with respect to the coordinate axes. 
In mechanics they are called the moments of inertia. Another popular mechanical quantity, the 


radius of gyration with respect to an axis, can also be expressed in terms of moments as 4 moz / mgo 


and 4/Myq / Mog , respectively. 


Another popular choice of the polynomial basis (x + iy)” (x - iy)’ where i is the imaginary unit, 
leads to complex moments 
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c= ll (x + iy) P(x - iy)" f (x, y)dxdy x 


GMs and CMs carry the same amount of information. Each CM can be expressed in terms of 
GMs as 


“pq ( 4 l Jeri ATT a 


and vice versa 


g p \( 1) (-1)17 
M ga ( kN 1 | gPa 41, p+q-k-l (4) 


CMs are introduced because they behave favorably under image rotation. This property can 
be advantageously employed when constructing invariants with respect to rotation. 


2.2. Orthogonal moments 


If the polynomial basis { Pij (x, y)} is orthogonal, i.e. if its elements satisfy the condition of 


orthogonality 


i P p, WIP anl; y)dxdy=0 6) 


or weighted orthogonality 


I wl, Y)P p, Y)Pinn(x, y)dxdy=0 (6) 


for any indexes p#m or q#n , we speak about orthogonal (OG) moments. G is the area of 
orthogonality. 


In theory, all polynomial bases of the same degree are equivalent because they generate the 
same space of functions. Any moment with respect to a certain basis can be expressed in terms 
of moments with respect to any other basis. From this point of view, OG moments of any type 
are equivalent to geometric moments. 


However, asignificant difference appears when considering stability and computational issues 
in a discrete domain. Standard powers are nearly dependent both for small and large values 
of the exponent and increase rapidly in range as the order increases. This leads to correlated 
geometric moments and to the need for high computational precision. Using lower precision 
results in unreliable computation of geometric moments. OG moments can capture the image 
features in an improved, non-redundant way. They also have the advantage of requiring lower 
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computing precision because we can evaluate them using recurrent relations, without 
expressing them in terms of standard powers. 


Unlike geometric moments, OG moments are coordinates of f in the polynomial basis in the 
common sense used in linear algebra. Thanks to this, the image reconstruction from OG 
moments can be performed easily as 


f(x, y)= X EM, Pyle, y) (7) 


2.3. Continuous moments 


Some of orthogonal moments are in terms of a continuous variable such as Legendre, Zernike 
and Gaussian-Hermite moments. All of these continuous moments depend on the same 
continuous polynomials. Here, we will discuss about such these polynomial functions and 
moments. 


2.3.1. Legendre moments 


There have been many works describing the use of the Legendre moments in image processing, 
e.g. references [29] and [30], among many others. The Legendre moments are defined as 


_ (2p+1)(29 +1) h 


Ag=— aI] POP CY) (x, y)dxd; p, q=0, 1, 2, (8) 


where P,,(x) is the n ™® degree Legendre polynomial (expression by the so-called Rodrigues’ 


formula) 
P a) =ar le?) (9) 


2Pp! dx? 


and the image f(x, y) is mapped into the square (-1, 1)x(-1, 1). The Legendre polynomials 


of low degrees expressed in terms of x ? are 


P (x)= >(3x2-1), (10) 


The relation of orthogonality is 
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IP, (x)P (dx =z TÔ (11) 


The recurrence relation, which can be used for efficient computation of the Legendre polyno- 
mials, is 


Po(x)=1, 


Pi(x)=x, (12) 


2p+1 
Ppa) EaP a) - P,a) 


2.3.2. Zernike moments 


Zernike moments (ZMs) were introduced into image analysis about 30 years ago by Teague [9] 
who used ZMs to construct rotation invariants. He used the fact that the ZMs keep their 
magnitude under arbitrary rotation. He also showed that the Zernike invariants of the second 
and third orders are equivalent to the Hu invariants when expressed in terms of geometric 
moments. He presented the invariants up to the eighth order in explicit form but no general 
rule about how to derive them was given. Later, Wallin [31] described an algorithm for the 
formation of rotation invariants of any order. Numerical properties and possible applications 
of ZMs in image processing among others. ZMs of the nth order with repetition / are defined 
as 


ai 


Ay= 


MAG 0)f(r, O)rdrd0, p=0, 1, 2, ...q=-p, -p+2, ..., p (13) 


i.e. the difference n - lIl is always even. The asterisk means the complex conjugate. The 
Zernike polynomials are defined as products 


= iq@ 
V gts 0) R,,(re (14) 
where the radial part is 
R,()= © Bork 
pal?) uit ves pak” a 
The coefficients 
y= A 
b = a6 


pak e ne 


can be used for conversion from geometric moments, 
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p+1 A Fa ( g ) Iq! 
A= a 2 j ( f jw'B ajiaji (17) 


7E 
k=lql,lql+2, ... j=0 1=0 


where 
-1; q>0 
= 18 
w | ; qs0 we 
The Zernike polynomials satisfy the relation of orthogonality 
2n1 z 
SIV g(t, OW g(r, OrdrdO= 73 STÔ bi, (19) 
The recurrence relation for the radial part is 
2pr 
R y) = FeqRpagi®)- = Rp24) (20) 


Computation of the Zernike polynomials by this formula must commence with 


Ropf)=Rp-pr)=r?” , p=0, 1, 


2.4. Discrete moments 
There is a group of orthogonal polynomials defined directly on a series of points and therefore 


they are especially suitable for digital images. Some of these polynomials such as Tchebichef, 
Krawtchouk, Hahn, dual Hahn and Meixner polynomials. 


2.4.1. Tchebichef moments 


The 2D TMs of order (p + q) of an image intensity function f (n, m) with size N xM is defined 
as 


ga Alp, N)AGQ, M)> 5. t,(n)t; (m) f (n, m) (21) 


n=0 m=0 


where t(n) is the p ™ order orthogonal discrete Tchebichef polynomial defined by [8] 


t,(n)= p! Eea, p- ra a d) (22) 


and 


B(p, N) 
A(p, N)= plp, N) 
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where f(p, N) is a normalization factor. The simplest choice of this factor is N ’ . The recur- 
rence relation of Tchebichef polynomials with respect to the chosen order is: 


(p+ Dt pln) - 2p +1)Qn-N +1)t,(n) + p(N?- p*)t,.(n)=0 (23) 


where p21 and the first two polynomials are tọ(n)=1 and ¢,(1)=2n - N +1. The orthogonality 


property satisfies the following squared-norm: 


pp, N)=(2p)!(9,.4) (24) 


2.4.2. Krawtchouk moments 


The definition of the n " order classical Krawtchouk polynomial is defined as 
n A ( 1 ) 
K,(x;p, N)=> akn pX =gb te -x;-N; y (25) 
k=0 


where x, n=0, 1, 2, ..., N, N>0, p€(0,1) and oF (0 ) is the generalized hypergeometric 


function 


œ (a) (b), z* 
„F(a, bic;z)=} O FT (26) 


and (x); is the Pochhammer symbol given by 


(x), = x(x + 1)(x +2) (x +k - 1) 


k21 and (x) =1 (27) 


The normalized and weighted Krawtchouk polynomials {K ,,(x; p, N)} are defined as [10] 


> w(x; p, N) 
K (x; p, N)=K,,(x; p, NW oat (28) 


where the weight function, w(e ) and the square norm, p(e ) are given as 


w(x; p, N)=(* )p*a- alae (29) 


and 


p(n;p, N)=(24)" a (30) 
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The normalized and weighted Krawtchouk polynomials have the following three-term 
recurrence relation: 


p(n - N)K a(x; p, N)=ALp(N - 2n) +n -xIK,(x;p, N)- Bn(1- p)K,.(x:p, N) (31) 
where 
Az (1 - p)(n +1) 
pN -n) 
(1 - pr(n +1)n 
B p(N -n)(N -n+1) 
with 


K (x; p, N)=vw(x; p, N) 
K(x; p, n)=(1 : px Wo; p, N). 


The 2D Krawtchouk moment of order (n + m) of an image intensity function f(x, y) with size 
N xM is defined as [16] 


Qun= 2 SK K(x Py N- 1)K (Yi Poy M- 1)f (x, y) y (32) 


The orthogonality property leads to the following inverse moment transform 


f(x, y) y =» S ¥ QumK, Ax; Py N- 1)K K „(y; Py M- 1) (33) 
If only the moments of order up to (N „ax > M max) are computed, then the reconstructed image 
in (2.30) can be approximated by 

ae „M, 

f(x, y)= T Py Q,mK nl; Py N - 1)K UE Po M- 1) (34) 


3. Formulation of geometric moments using digital filters 


This section first reviews the generation of GMs using digital filter structure as proposed by 
Hatamian [19]. This is then followed by a review of the improved version used by Wong and 
Siu [20]. 
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3.1. Hatamian’s model 


Hatamian proposed the all-pole digital filter structure to compute GMs up to the 16" order. 
The one dimensional GMs of order p for a N -length sequence x[n] is defined in this model 
as 


m,= X n?xtn] (35) 


n=0 


One reason for using digital filters as a moment generator is based on the convolution of the 


aforementioned sequence with impulse response, n Pu[n], where u[1] is the unit step function. 
Hence, the output of the digital filter y, evaluated at the point n = N - 1 can be expressed: 


YINI- Y xIkIN =)? (36) 


where x[n] is the reversed sequence. Figure 1 shows the structure of a single pole digital filter 


with transfer function = , which is equivalent to an accumulator with unity feedback. This 


z-17 
accumulator has a delay in the feed-forward path. For p cascaded filters, the corresponding 
transfer function is given by 


He) =g" 67) 


LOH- 


Figure 1. Single-pole filter structure [19] in feedforward path. 


The 1D cascaded filters structure is shown in Figure 2, where H,(z)== 1 T: 


x[n] Yo [n] yı [n] Yp- [n] Y [n] 
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Figure 2. Cascading of single-pole filters for generating of moments up to order p . 
The relationship between GMs and the all-pole digital filter outputs are related by the 
following expression 


na E CV (38) 


where y, is the r " digital filter output and C, , is a matrix of coefficients directly obtained from 


pr 


the impulse responses of the all-pole digital filters as given in [27]: 


0 y r>p 
Cpr= (-1)? ; r=0, p20 (39) 
tara = (r 7 NC ay ; r>0, p>0 


For a two dimensional image, the relationship between the GMs and digital filters outputs can 
be expanded to 


tg Ci gal (40) 


However, the all-pole filter structure has delays in the feed-forward path; hence it causes an 
increase in the computation time. 


3.2. Wong and Siu’s model 


To overcome this problem, Wong and Siu [20] moved the delay unit to the feedback path of 
the filter, as shown in Figure 3. The transfer function is given as 


H,(z)=(—44)"" (41) 


Jaza! 


x[n] y[n] 
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Figure 3. Single-pole filter structure [20] in feedback path. 


In this case, however, the GMs are obtained from the digital filter outputs and a different matrix 
coefficient, as shown below: 


m,= Dh (42) 
where the coefficients of D p,r are obtained from the following recurrence formula shown in [23]. 


0 ; p>0,r=0 
Dag 1 7 p=0, r=0 (43) 
MD saga 7 D par) ; others 


For a two dimensional image, the relationship between the GMs and the improved digital 
filters structure outputs are related by 


hig 2 Dad Dates (44) 


3.3. Proposed method based on the lower output values of digital filters 


We begin with the one-dimensional case, where the results can be easily extended to two- 
dimensional. Consider a digital filter with impulse response h pL = nun], where u[n]is the 
unit step function and p is the moment order. Now, assume the input of the digital filter is 
given as x[n] as mentioned in (3.2). Based on the convolution theorem, the output, y[n] is 
therefore 


y,inJ=x[n}*h [n] (45) 


Using the digital filter as shown in Figure. 3 and changing the unit step function to u[n + p] to 
accommodate the sampling of the digital filter outputs at earlier instances, we get the following 


yplnaxtn3(" 5” Jute pl =È xt") (46) 
Substituting, n=N - p yields 
YIN - pl=d xt", *) (47) 


Expanding the binomial equation, and using the Stirling numbers we get 
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1 Np 5 
VIN - pl= ard, È sip, DIKAN -ky (48) 


where s,(p, i) are the Stirling numbers of the first kind [28], which satisfy 


(N -k)! J ; i 
wrp dl, DUN -K (49) 


Using (3.2), we can rewrite (3.15) in terms of GMs as follows: 


¥ LN B pl= ay sı(p, i)m; (50) 
0 


i= 


Now by taking the inverse of (3.16), the GMs can be obtained in terms of the digital filter 
outputs thus: 


m=} rsp, r)y,LN -r] (651) 


where s,(p, r) are the Stirling numbers of the second kind [28], and the Stirling numbers of the 


first and second kind can be considered to be inverses of one another: 


w (- 1) Pts (p, i)s,(r, p) = Sir i 


p=0 
where 6,, is the Kronecker delta. 


Notice now, for the p order, it can be shown that the digital filter outputs are sampled at 
N - p, unlike the previous works which were sampled at N or later instances of N [19]-[20]- 
[21]. As the order ta is reached, the digital filter output values begin to decrease. This allows 


the use of low value digital filter outputs for the formulation of GM. The 2D moments can be 
obtained by expanding the 1D model for the digital filter outputs as follows: 


My =>) ¥ r!s!s,(p, r)s(q, s)y,LN -1, N -s] (53) 
r=0 s=0 


3.4. Experimental studies 


A set of experiments were carried out to validate the theoretical framework developed in the 
previous sections and to evaluate the performance of the proposed structure. This section is 
divided into 3 parts. In the first subsection, an artificial image of size 4x4 is used to generate 
GMs up to third order. The computational complexity of three algorithms — the algorithms of 
[19], [20] and the proposed method — is then analyzed and discussed in the second subsection. 
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In the third subsection, the speed of the proposed method is compared with the speed achieved 
using [19] and [20]. 


3.4.1. Artificial test image 


rtificial test image of size 4x4 was used to prove the validity of the proposed approach. In this 
case, the digital filter outputs up to third order were generated. The intensity function of the 
test image is given in the following matrix: 


111 114 103 116 
109 101 113 108 
106 102 112 104 
107 110 105 115 


xim, n]= 


The difference between the digital filter output values for [20] and proposed structure is shown 
in Table 1. It is clear that for orders higher than one, the proposed output values are much 
lower than [20] as the order increases. 


Filter structure output values 


Digital filter outputs [20] Proposed structure output values 
Yoo 1736 1736 
You 4326 4326 
Yo2 8651 4325 
Yo3 15148 2172 
Yio 4358 4358 
Vii 10870 10870 
Yiz 21750 10880 
Y20 8742 4384 
Yu 21811 10941 
Y30 15331 2205 


Table 1. A comparison of filter output values between [20] and the proposed method for an artificial image, x[m, n], 
up to third order. 


3.4.2. Computational complexity 


The advantage of the proposed method lies in the smaller digital filter output values as 
compared to [19] and [20]. However, it would still be useful to study the computational 
complexity of these three methods and the direct method in term of the number of additions 
and multiplications. The proposed method, [19] and [20] consist of two main steps. Digital 
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filter outputs are obtained from the respective digital filter structure. Then, these outputs are 


linearly combined to compute GMs. 


For a grayscale image of size N xN and GM up to order of s , where s = (p +g) , the number 


of additions and multiplications for the proposed method, [19] and [20] are shown in Table 2. 


Aisne Additions Additions (Digital filter outputs © Multiplications (Digital filter 
orithm 
g (Digital filter stages) to GMs) outputs to GMs) 
4 3 2 

[19] s DIN +242) 41) seis? AL steal 

[20] ta Din + 222) sh Net vas 1a) geile esset 
Proposed (s + 2)(s - 3) sls - 52435 + 14) sls - (52435 + 14) 
method “t DINN +2)-=====| Sotstirit eee as 


Table 2. Complexity analysis of GMs computation using digital filters for image of size N xN and maximum order of 
s=p+q. 


It can be seen that even though the complexity of the linear combination stage for the proposed 
method is the same as [20], there is saving in the number of additions at the digital filters stage. 
This can be clearly shown in the example below. For N = 512 and s=45 , the number of 
additions needed in the filter stage for [20] is 12612096 while the proposed method just requires 
12090594 additions. The summary of the complexity comparison for all the three methods to 


compute GMs up to 45" order is shown in Table 3. 


Algorithm Additions Multiplications 
[19] 12847524 210795 
[20] 12791451 179355 
Proposed method 12269949 179355 


Table 3. Complexity analysis of GMs computation using digital filters for image of size 512x512 ands=45. 


For a 128x128 grayscale image, the advantage of the proposed filter structure as compared to 


[19] and [20] is clearly depicted in Figure 4. 
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a 


Number of Additions 


Figure 4. Number of additions for a 


40 60 80 
Moment Order 


28x 128 grey-scaled image. 


3.4.3. Speed performance and comparison studies 


The computation speed of the proposed method is compared with [19] and [20]. CPU elapsed 
time is used in the evaluation process in all the performed numerical experiments. The codes 
were all written in MATLAB7 and simulations were run with a 3GHz Intel Core2 with 2GB 


RAM and the average time is used in the discussion. The images used for the experiments were 


the Pepper and Lena images, 


Table 4 shows the simulation time for GMs computation on the Pepper image of size 128x128 


for orders of 5 to 55, in a step of 10. The same experiment was repeated on the Lena image of 


as shown in Figure 5. 


size 512x512. The simulation times are shown in Table 5. 


100 120 


Moment order (s=p +q) [19] [20] Proposed method 
5 0.4 0.39 0.388 
15 1.12 1.104 1:05 
25 2.03 1.97 1.81 
35 3.28 3.14 2.82 
45 5:15 4.87 4.32 
55 7.96 7.48 6.63 


Table 4. CPU elapsed time in milliseconds for the 128 x 128 Pepper's test image. 
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Moment order (s=p +q ) [19] [20] Proposed method 
5 6.03 6.02 6.00 
15 16.264 16.222 16.018 
25 26.817 26.722 26.13 
35 37.866 37.677 36.494 
45 49.663 49.325 47.343 
55 62.542 61.982 58.99 


Table 5. CPU elapsed time in milliseconds for the 512x512 Lena's test image. 


The tables clearly show that the proposed method requires less time than [19] and [20] to 


compute GMs of the same order. They also show that the time saving increases as the order 


of moments increases. In Figure 6 and Figure 7 we provide a comparison of required CPU time 


between [19], [20] and the proposed method. 
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(a) (b) 


Figure 5. Test images: (a) Peppers and (b) Lena. 


8 
TS 9] 

7 —=— [20] 
—v— Proposed 


CPU Elapsed Time(mS) 
A 


5 10 15 20 2 30 35 40 45 50 5 
Moment Order 


Figure 6. Linear scale of CPU time in seconds for the 128 x 128 gray-scale peppers image. 
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70 
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—v— Proposed 


50 
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30 
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20 
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Moment Order 


Figure 7. Linear scale of CPU time in seconds for the 512x512 gray-scale Lena image. 


4. A Reduced 2D digital filter structures for fast computation of geometric 
moments 


In the previous works, the linear transformation is performed using an external PC to relate 
the outputs of the digital filter structure to generate the moments. However, finding of a 
relationship between the geometric or orthogonal moments and the digital filters outputs has 
been stood as the main meaning of similar works [19]-[20]. 


In all the above aforementioned literatures, the basic concept involved the computation of 
geometric and other types of moments using the accumulator grid structure has remain 
unchanged over the past decades. 


First we summarize the designing modality of all-pole digital filters using 1-D and 2-D Z- 
transforms [32]. We believe that relevance creation manner among the real-time and Z-domain 
is one of the most essential issues in studying of digital image moments. This section presents 
the results of a study aimed at the formulation of a new method to reduce the filter resources 
used in the digital filter structure. This study, though not exhaustive, serves to provide a 
suitable classification framework that underlies a new approach in digital filter design for the 
computation of moments. 
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4.1. Reduced digital filter structure 


Unlike the previous researchers who used a 1-D Z-transform to derive a 2-D digital filter 
structure by cascading the filters both in the rows and columns. However, we use a 2-D 
definition of Z-transform to obtain the impulse response of the filter which led to a reduced 
digital filter structure as compared with [19]. The 2-D Z-transform for the image, f[m, n] is 
given as: 


too +00 


F(z, 2)=2, >, fim, n]z,"29" (54) 


m=1 n=1 


The impulse response for a 2-D image becomes: 


hy, lm, nj=( et 4 Julm] nra utr] (55) 


and the transfer function H 7 jEr z) , in the 2-D Z-transform domain for this filter structure is 


shown as 


1 


H, (21 Zy)= m- z) z Ze (56) 


Using the above transfer function, the relationship between the input and the output of the 
digital filter is: 


x (2, 2) 


Y par z)= 1- z1) P" (1 - z!) (57) 
Based on (4.4), the zero order of the digital filter output is derived as 


X (zy 2) 


Yoo(Zv 2y)= 1 a 1 Zt (58) 


Thereafter, a recurrence relationship between the previous and the next outputs of each digital 
filter for the row and columns as shown in Figure 8 can be obtained: 


Y 7 (z poe ) 

Y piale 2)=— gs (59) 
Y " (z ,z ) 

Y p get (2 Zp) = I a a (60) 


By taking the inverse 2-D Z- transform of (4.6) and (4.7), we get the following: 
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Ypq Ypo+1q 


Yp,qt+1 


Figure 8. Recurrence relationship between the digital filter outputs of the row and column. 
Ypsi N]EY prq- 1, nl]+y, lm, n] (61) 
Yp geil, n]=Yp gam, n-1]+ y, lm, n] (62) 


The implementation of proposed digital filter structure for generating moments up to (p +g) 


orders is shown in Figure 9. 


} a T 
y -7 
z. Yia V21 ra] iii Ypı 


Y22 val wee p2 
w| i Y23 ra| n Vp3 


x[m,n] 


Yoa Yia Yza aal Ne Ypa 


Figure 9. Reduced digital filters for generating the 2-D geometric moments up to (p + q) order. 
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Figure 10. Digital filters for generating the 2-D geometric moments up to (p + q) order used in [2] and [8]. 
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Yp 


Ypı | 
Yoe| 


As compared to the digital filter structure used in [19] and [20] as shown in Figure 10, it can 
be seen that the difference is the prefix, y, and y, used in the rows. In the proposed model, 


the outputs Yoo, Y10 rv Y po Occur at the row filter, unlike in [19] and [20], they occur at the 


column filters. Hence, the difference in savings of digital filters used in the proposed method 
is determined by the maximum p order. for example, in the design of the 40" order geometric 
moments, where the maximum p order is 40, and thus the number of digital filters used in 


proposed method will be 40 digital filters less than [19] and [20]. 


In our proposed method, we begin by showing the relationship between the digital filter 
outputs and the geometric moments by considering a 1D image. So, the first few outputs of 


the digital filter are 


yolN =} KIEN -Kl= 


yilN I= x(kJh LN -K]= È afk UN -k+1)=(N + Dily- by 


yo J= 3 KIKAN -K]= È xf] Z(N -k + D0 -k +2)=3(N + DON +2)uo- ZN +341 + 3H 


(63) 


Hence by solving them, the above geometric moments can be obtained in terms of the digital 


filters outputs. 
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Ho= Yo 
H=(N + Dyo- y (64) 
Hy=(N +1)°yo- (2N +3)y, +2y, 


A matrix notation showing the above relationship between the geometric moments and the 
digital filter outputs as expressed 


e=CuYp (65) 


where [lp is the geometric moment vector, Y p is the digital filter output vector, and Cy is a 


square matrix that a recurrence relationship between this matrix elements is given as 


0 ; psr 
(-1)”p! o par 
Cp =|(N +1)? b p (66) 
a Coak 
pC nai? ps. m oth. 


PER 


where, | © | represents the floor function. 


Following the same procedure used in obtaining the 1-D relationship digital filter outputs and 
geometric moments, it can also be extended for 2-D image. This is done by taking the transpose 
of the digital filter outputs and the transpose of the matrix, Cy . Fora 2-D image of dimensions 
x N , the geometric moments can be obtained from 


Hp=CmY wen” (67) 


Also, the summation forms of (4.12) and (4.14) can be written as: 


H=? Cord (68) 
Hm= È ECC, Yor (69) 


4.2. Experimental results 


In this subsection, we will begin with an example to determine the geometric moments up to 
third order for 2-D image using the proposed and existing method [19]. Figure.3. shows the 
proposed digital filter structure and the structure used in [19]. As can be seen from Figure 11(a) 
and (b), the proposed filter structure used three less digital filters. The difference between them 
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is maximum p- order. As the number of moment orders increases the savings of the digital 
filters will be large. For example, the geometric moments used are forty orders, and then the 
number of digital filters will be reduced by forty as compared with existing methods. 


x" [m,n] Yoo Yao Yeo V0 
ee 


x[m, n] 
>| 


Yoo Vio J20 J30 


Figure 11. Digital filter structure for generating up to third order geometric moments for 2-D image. (a) proposed 
model and (b) existing model. 


4.2.1. Artifical test image 


Artificial test images of small size are used to prove validity of the proposed architecture. To 
illustrate the workings, an artificial image with size of 2x3 to generate up to third order of 
geometric moments is considered. The intensity function of the test image is represented by 
the following input matrix: 


| 


xim, n=; 41 


Consider the dimension of the input image is 2 x3 , it means the endpoint of the filter outputs, 
is 6. Therefore, all generated outputs by the row filters are defined at different intervals till 
6, where be represented by the Table 6. Table 7 shows the generated outputs by the column 
filters for two sampled seconds (3,6). The input image must be reversed in this method and 


denoted as x È. 
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n 0 1 2 3 4 5 6 
xf 0 1 4 2 6 3 5 
Yo 0 1 5 7 6 9 14 
yı 0 1 6 13 6 15 29 
V2 0 1 7 20 6 21 50 
V3 0 1 8 28 6 27 77 


3 6 
28 TI 
28 105 


Table 7. State table method used in [8] for obtaining column filter outputs. 


The highlighted outputs can be collected as Y matrix in (4.12) and the coefficients matrix 
( D ) as defined in (3.9), will generate the third order moments’ set of the artificial image as 
follows: 


Ho=21, Ho=42, Ho=98, Hoz=252 
Hyg= 28, 41595, (h2=125 
H2542, by) =81 


(70) 


H30=70 


If we derive the moments of the same example with the proposed method, the endpoint of the 
procedure is also 6. However, the results of the digital filter outputs are listed in Tables 8 and 
9. In this case, according to Figure 11(a), it is well-known that the number of the digital filter 
is deduced up to maximum p- order. Furthermore, in this method it does not need to reverse 
of the input sequence. 


n 0 1 2 3 4 5 6 
EL 0 5 2 3 4 6 1 
y 0 5 7 3 7 6 7 
yl o 5 3 6 7 7 7 
Yoo 0 5 8 14 7 14 21 
Yro 0 5 13 27 7 21 42 
Yoo 0 5 18 45 7 28 70 
30 o 5 23 68 7 35 105 


Table 8. State table of proposed method for obtaining row filter outputs. 
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n 3 3 6 
Yoo 14 68 105 
Yo. 14 
Yor 14 
Yo 14 


Table 9. State table of proposed method for obtaining column filter outputs. 


Based on (4.13) and (4.14), we can derive the digital filter outputs’ matrix ( Y) and the 
coefficients matrices ( Cy, Cy ), respectively. Then, the set of third order moments are 


obtained as (4.17). 


4.2.2. Real test image 


The images used for the experiments were the Pepper (128x128) and Lena (512x512) images as 
shown in Figure 12(a) and (b). As shown in Figure 13(a) and (b) the number of additions for 
digital filter processes are compared in [19] and the proposed method. 


(a) (b) 


Figure 12. Test images (a) Pepper and (b) Lena. 


183 


184 Digital Filters and Signal Processing 


—— [5] Approach |: 
— -—- Proposed : 


—— [5] Approach 
H- — - — ‘ Proposed þess 


Number of Additions 
Number of Additions 


0.5 i shee seneteeedensennesened 


i i i i i i 0 i i i i i 
0 
0 20 40 60 80 100 720 740 0 100 200 a oe 400 500 600 
Moment Order foment Order 


(a) (b) 


Figure 13. Comparison of the number of additions in digital filter part for [19] and proposed method for (a) Pepper 
image (b) Lena image. 


5. Conclusions 


Orthogonal moments are widely used in image analysis and as pattern features in pattern 
classification. The computation of orthogonal moments can be achieved via geometric 
moments. Hatamian introduced cascaded digital filters, where each filter operates as an 
accumulator to generate geometric moments. 


One of the weaknesses in using the outputs of the cascaded digital filters to generate the GMs 
is that the filter outputs increase exponentially as the orders of the moments increase. This 
work proposes a new formulation to solve this problem by sampling at earlier instances of 
N, N -1, ....N +p, where p is the maximum moment order for an N x N image. This step 
then paved the way to use a set of lower digital filter output values. The work demonstrates 
the efficacy and validity of the new algorithm in two ways: one, by comparing its complexity 
of computation with the complexity of two state of the art models proposed by Katoulos and 
Andreadis [21] and Wong and Siu [20] and two, by carrying out a set of experiments on speed 
of computation comparing the results obtained using the proposed method with those 
obtained using existing methods. A number of findings indicate the superiority of the 
proposed method: (1) A savings of as much as 45% is achieved for the proposed method in the 
number of additions as the moment order approaches N when compared with the existing 
methods and (2) this leads to less computational time for the proposed method to derive the 
GMs. This work has focused on the software implications of the algorithm. If the proposed 
method is implemented in FPGA or ASIC based platforms, a great savings in terms of bit- 
widths will be realized. 


Also we showed a reduced method where the number of digital filters used in the generating 
of geometric moments is reduced by p -order when compared with the existing methods. The 
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proposed method is modeled using the 2D Z-transform, and the theoretical framework for the 
proposed reduced digital filter structure is developed and the experimental results validate 
the performance. 
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1. Introduction 


Many digital signal processing (DSP) systems tend to have a very high computational 
complexity when they target a large part of the Nyquist band. This corresponds to 
a wide-band system with one or several so called don’t-care bands approaching zero. 
Examples of such systems include frequency selective filters, fractional-delay filters, and 
differentiators. This chapter considers finite-length impulse response (FIR) filters due to 
their attractive implementation features. In particular, they can be implemented with 
non-recursive structures. In contrast to infinite-length impulse response (IIR) filters, they 
are therefore always automatically stable and have no bound on the maximal sampling rate, 
see [1, 2]. 


For frequency-selective wide-band FIR filters, the frequency-response masking (FRM) 
technique can be employed for complexity reductions due to its use of sparse (namely 
periodic) subfilters, see [3-10]. For other functions, the FRM technique cannot be used 
directly, and one therefore has to seek other methods to reduce the complexity. This chapter 
discusses such a method which utilizes a two-rate technique, but only for the derivation of 
efficient single-rate structures. The basic two-rate approach was originally introduced in [11] 
and has since then been exploited and extended for various contexts as detailed in [12-19] 
and to be reviewed in this chapter. For single-function systems, it is however necessary 
to combine the two-rate technique with the FRM approach in order to achieve an overall 
complexity reduction. For multi-function realizations, complexity savings may be obtained 
without incorporating the FRM approach but it offers further complexity savings in such 
cases, as exemplified in [19]. Recent results have shown that the two-rate approach offers 
dramatic complexity reductions for wide-band systems, especially when combined with the 
FRM approach. 
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1.1. Chapter outline 


Following this introduction, Section 2 considers the two-rate based structure that is 
appropriate for so called left-band and right-band systems which have don’t-care bands at 
the low-frequency and high-frequency regions, respectively. Section 3 discusses the extension 
to so called mid-band systems which have don’t-care bands at both the low-frequency and 
high-frequency regions. In Section 4, multi-function system realizations are considered, 
whereas Section 5 gives more implementation details. Finally, Section 6 concludes the 
chapter. 


2. Two-rate based structure for left-band and right-band systems 


This section will first revisit FIR filters and their computational complexity. After that, the 
two-rate based structure for left-band and right-band systems will be discussed. 


2.1. Complexity of FIR filters 


Consider a causal FIR filter with an impulse response h(n), transfer function 


He) = }_ h(n)”, (1) 
and frequency response 


(ei) = Y nfe Fen, @) 


n=0 


The order of the system is Ny and the impulse response duration (length) is Ny +1. A 
direct-form implementation of the filter, corresponding directly to the convolution 


Nu 
y(n) = } x(n — k)h(k), (3) 
k=0 


where x(n) is the input and y(n) the output, requires Ny +1 multiplications and Ny 
additions to compute each output sample y(n). In the case of a linear-phase frequency 
response, h(n) is symmetric or anti-symmetric which reduces the number of multiplications 
to roughly Ny /2 1 


The filter order required is determined by the application and specification. For example, 
for frequency selective filters, the order is inversely proportional to the transition band 


1 An even-order symmetric (anti-symmetric) linear-phase filter requires Ny /2 +1 (Ny /2) multiplications whereas an 
odd-order linear-phase filter requires (Ny + 1) /2 multiplications. 
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(don’t-care band) A = ws — we, where we and ws denote the passband and stopband edges, 
respectively, see [20, 21]. Hence, when the don’t-care band decreases towards zero, the order 
increases rapidly. Then, using a direct-form realization, the computational complexity may 
become intolerable as it follows the filter order. The same trend exists also for other functions 
that are not frequency selective filters, like differentiation and integration, as seen in [22]. 


2.2. Two-rate based structure 


To reduce the complexity, we consider here a structure that is derived via a two-rate 
approach, seen in Fig. 1. This structure is efficient for left-band systems (like a differentiator) 
targeting the frequency region w € [0, we], 0 < we < 7. The same structure can also be used 
for right-band systems targeting the band w € [we, 71], 0 < we < 7. The only difference will 
appear in the design, and we will therefore focus on the left-band case in this chapter, and 
only comment upon the right-band case in the design section. 


For a left-band specification, the basic idea is to first interpolate the input signal x(n) by 
two through upsampling by two followed by a lowpass filter with transfer function F(z) 
2. Then, a subsequent filter with transfer function G(z) follows that performs the actual 
function. Finally, downsampling by two takes place to retain the original sampling rate. 
Using multi-rate theory, see [23], it is readily shown that this scheme corresponds to a linear 
and time-invariant (LTI) system with a transfer function H(z) that equals the Oth polyphase 
component of the cascaded filter F(z)G(z), i.e., 


H(z) = Fo(z)Go(z) + 2-1Fy(z) Gi (z) (4) 
where 
F(z) = Fo(z*) +271 (2°) (5) 
and 
G(z) = Go(z*) +z !G,(z). (6) 


The final realization is thus a single-rate structure. A two-rate technique is only used to 
derive efficient structures. It is noted here that the order and delay of the overall filter H(z) 
is Ny = (Nf +No)/2 and Dy = (Dp + Deg) /2, respectively. This can be understood by 
noting that F(z) and G(z) can be viewed as operating (in principle) at two times the input 
rate, because the structure is derived by sandwiching F(z)G(z) between upsampling and 
downsampling by two. 


? The same function can be achieved by sampling the underlying analog signal with a higher sampling rate instead of 
sampling it slower and then use interpolation in the digital domain. However, this also increases the requirements 
on the analog-to-digital converters which are power-hungry components and in many cases one of the bottlenecks 
in overall systems. It is therefore often preferred to perform interpolation in the digital domain. 
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Figure 1. (a) Two-rate approach. (b) Equivalent LTI system when F(z) is an HB filter. (c) and (d) Polyphase component Fo (z) 
of the HB filter F(z) = Fo(z*) + z7PF when realized with the FRM approach for an even-order (c) and odd-order (d) masking 
filter. 


2.2.1. Filter Types 


It is possible and efficient to let F(z) be a linear-phase half-band (HB) FIR filter °. Such a filter 
has a symmetric impulse response and every second impulse response value is zero, except 
the center tap which equals unity for an interpolation filter that preserves the signal energy. 
This corresponds to a pure-delay polyphase component F; (z), namely F; (z) = z~ (Pr-1)/2, 


3 If the delay is of importance, one may need to use a nonlinear-phase (approximately linear-phase) low-delay HB 
filter F(z) instead. Further, if there are additional requirements in the don’t-care band, like attenuation requirements 
at w = 71, a general filter F(z) must be used instead, i.e., a non-HB filter. See [19] for details. 
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Overall filter H(z) Half-band filter F(z) G(z) 
Type I, even order 2(m+p+1) Type I, order 4m +2 Type I, order 4p + 2 
Type IL, odd order 2(m+p)+1 Typel,order4m+2 Type I, order 4p 
Type III, even order 2(m + p + 1) Type I, order 4m + 2 Type IIL, order 4p + 2 
Type IV, odd order 2(m + p)+1 Typel,order4m+2 Type IIL, order 4p 


Table 1. Linear-phase filter types. 


where Dr is the delay of F(z), which is always an odd integer. When F(z) is a linear-phase 
filter, G(z) is of the same type as that of the overall filter H(z), i.e., a linear-phase filter 
(nonlinear-phase filter) when H(z) is a linear-phase filter (nonlinear-phase filter). In this 
section, we focus on linear-phase filters. In the next section, nonlinear-phase applications are 
considered. 


When F(z) is a linear-phase HB filter, it is a symmetric Type I filter with the odd-integer delay 
Dr. Its delay contribution, Dr /2, to the overall delay Dy is therefore an integer plus a half. 
It is consequently the delay contribution Dg /2 of G(z) that determines whether the overall 
delay is an integer or an integer plus a half. As Dg/2 then must be an integer or an integer 
plus a half to obtain an overall linear-phase filter, Dg must be an integer. Consequently, G(z) 
is either a Type I or Type III linear-phase FIR filter, i.e., an even-order filter with a symmetric 
or anti-symmetric impulse response. In other words, the type and order of G(z) determines 
the type and order of the overall filter, as summarized in Table 1. A formal proof of these 
facts is given in [17]. 


The order of G(z) is thus somewhat restricted as it cannot take on all even orders. However, 
the effective order of G(z) can be reduced by two by setting its first and last impulse response 
value to zero. In this way, two multiplications and additions may be saved in some cases. 
For the HB filter F(z), it does not make sense to try to reduce the effective order in this way, 
as its impulse response is always zero for odd indexes of n (except for the center tap). 


2.3. Complexity reduction 


Assume that H(e/”) is to approximate a desired function D(jw) in the band w € (0, wc]. Due 
to the principle of interpolation by two in the two-rate based scheme, the effective bandwidth 
of G(z) is wc/2, and thus always less than 77/2. The complexity of G(z) alone will therefore 
be substantially lower than that of a regular direct-form realization of H(z). (This will be 
discussed in more detail in the design example considered later in Section 2.4). However, 
the overall complexity is also determined by the filter F(z). The requirement on this filter is 
roughly the same as that of the overall filter H(z) and its complexity is therefore relatively 
high. In other words, a major part of the overall complexity is moved to the filter F(z) 
and thus to Fo(z) in Fig. 1. Therefore, for a single-function system, there will not be any 
computational savings using this approach straightforwardly. This is because we can equally 
well combine the three subfilters into one single conventional filter. 


Nevertheless, overall savings can indeed be obtained by utilizing additional complexity 
saving techniques for the lowpass frequency selective HB filter F(z). Specifically, by realizing 
F(z) as an FRM filter, see [3-10], we can express the transfer function as 


F(z) = 2A(z")Bo(z) + 2[z-*?4 — A(z*)]By (z) (7) 
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where A(z") is a period model filter and [z~'?4 — A(z!)] is its complement, whereas Bo (z) 
and By (z) are masking filters. Specifically, in the case of a HB filter, as detailed in [5, 7], A(z) 
is given as 


A(z) = Ao(z*) +. 0.5274, (8) 


with D4 being the delay of A(z), whereas the masking filter are related according to 
By(z) = 2-8 — (—1)?®Bo(—2), (9) 
with Dg being the delay of Bg(z). One then finds that Fy(z) becomes, for Dg even: 
Fo(z) = 22° @P4*0/2Boy (z) + 2[2Boo(z) — z 78/7] A(z") (10) 
and, for Dg odd: 
Fo(z) = 22 Pa -0/2 Boo (z) + 2[2Bor (z) — 27-0] A(z") (11) 


where Boo(z) and Bo;(z) are the polyphase components of Bo(z), i.e., Bo(z) = Boo(z?) + 
z~!Bo,(z*). The resulting structures for Fo(z) are depicted in Fig. 1(c) and (d). More details 
can be found in [7]. 


As seen, Fy(z) makes use of three subfilters, of which Ag(z") is periodic for an integer 
L > 1. A periodic filter is a sparse filter, meaning it has many zero-valued filter coefficients. 
Specifically, only every Lth impulse response value of Ag(z") is non-zero. Consequently, 
a linear-phase filter A(z’) of order N4 requires roughly only N4/(2L) multiplications and 
N,/L additions. In this way, substantial overall savings can be obtained as compared to the 
conventional direct-form structures. 


2.4. Design 


Filters are typically designed in the minimax (Chebyshev) sense or least-squares sense, or 
possibly combinations thereof, see [24-26]. The goal of this chapter is to demonstrate that 
the complexity (number of multiplications and additions) can be reduced when using the 
two-rate based structures instead of regular structures. This will be done by designing both 
filter classes to meet the same specification and then comparing the resulting complexities sA 
To this end, the selection of approximation type is irrelevant, as long as one uses the same for 
both filter classes. In this chapter, we use minimax design, but other designs can of course 


be used as well after some minor appropriate modifications. 


For minimax design, the maximum of the modulus of an error function E(jw) is minimized. 
The error function is typically given as 


E(jw) = W(w)[H(el) — D(ja)], w € 0. (12) 


* Another type of comparison is to study the approximation error differences between two solutions having the same 
filter implementation complexity. However, such a comparison is appropriate when using two different design 
methods applied to the same filter class (structure) which does not apply here. 
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where D(jw) is a desired function to be approximated in the frequency band Q by the filter 
frequency response H(e/“’), whereas W(w) is a positive weighting function. A conventional 
FIR filter, with the frequency response in the form of (2), is then designed by solving the 
following approximation problem. 


Approximation problem: Given Ny, find the unknowns h(n) and 6 to minimize 6 subject to 
|E(jw)| < ô. (13) 


For a linear-phase filter, we also have the additional symmetry constraints h(n) = A(N — n) 
or h(n) = —h(N —n). 


For a conventional filter, the problem above is a convex optimization problem which has 
a unique global optimum. It can be found using linear programming, see [27], or the 
more efficient McClellan-Parks-Rabiner algorithm given in [28]. In practice, one usually 
has a specification on the desired approximation error ô, say ôe. The filter will meet this 
specification if ô after the optimization satisfies ô < ôe. 


For the two-rate based filters, the design becomes more intricate because it contains cascaded 
and parallel subfilters. This means that the unknowns are not h(n) but instead f(n) 
and g(n), in general, and a(n), bo(n), and g(n) when F(z) is realized as an FRM filter. 
Hence, conventional design methods can no longer be used. Moreover, due to the cascaded 
subfilters, we are now facing a nonlinear (nonconvex) optimization problem, which means 
that an overall globally optimum solution cannot be guaranteed. Nevertheless, if carefully 
designed, even a locally optimum solution for a two-rate based structure can be substantially 
less complex than the corresponding globally optimum direct-form structure. To ensure a 
good local optimum, the overall two-rate based filters are designed in three steps as explained 
below. Although F(z) should here be an FRM HB filter in order to achieve any savings, 
we will first explain the essential design steps in terms of a regular HB filter for the sake 
of simplicity. After that, the necessary modifications required for an FRM design will be 
pointed out. 


2.4.1. Basic Three-Step Design Procedure 


Given the desired function 
D(jw) = eH Net+NF)/4D 9 (ja) (14) 


and bandwidth w € [0, wec], wc < 7, as well as a targeted approximation error ĉe, perform 
the following three-step procedure for each combination of filter orders Ng and Nr around 
estimated required orders Nc and Nr: 


(1) Design the regular FIR filter G(z), which gives Go(z) and G,(z) after polyphase 
decomposition. It is done by minimizing the maximum of |Ec(jw)| in the band w € 
(0, w¢/2] where 5 


Ec (jw) = G(el”) — eJ#Ns/2 Dy (j2w). (15) 


5 For a right-band specification, the band for G(z) is w € [w,/2, 7/2]. 
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(2) Design a regular lowpass HB FIR filter F(z), which gives Fy(z) and F,(z) = z~(PF-1)/2, 
Dr = Nf /2, after polyphase decomposition. It is done by minimizing the maximum of 


|Er(jw)| in the band w € [7 — w¢/2, 7], where ©, 


Er(jw) = F(e!”). (16) 


(3) Use F(z), Go(z), and G;(z) obtained above as the initial solution in a further nonlinear 
optimization routine that solves the approximation problem stated in (13). If the resulting 
approximation error ô is smaller than ôe after the optimization, store the result. 


The estimated orders required, Ñg and Nr, can be found by separately designing G(z) and 
F(z) to approximate their respective desired functions (as given in Steps 1 and 2, respectively) 
with the same tolerance as the overall targeted error, i.e., ĉe (or similar as in [17-19]). As the 
bandwidth of G(z) is always below 7/2, its order is typically below 12 for approximation 
errors down to some —100 dB, provided a smooth function is targeted, like a differentiator 
or integrator. Hence, the value of Ng is readily found by designing G(z,d) for all low-order 
filters, using conventional techniques, and then set No to the lowest one for which the 
approximation error |Eg(jw)| is below ôe. As to the lowpass HB filter F(z), the value Ne 
can be found via well-known formulas for order estimation, see [20, 21], and a few designs 
around the estimated value. 


Regarding the designs, the problems in Steps 1 and 2 are convex, and thus have unique 
global optima, provided they are formulated in accordance with the approximation problem 
stated earlier in this section. These problems can be solved using any regular solver for such 
problems. As F(z) is a linear-phase filter, it can alternatively be designed using the efficient 
McClellan-Parks-Rabiner algorithm given in [28]. The problem in Step 3 is nonlinear because 
of the cascaded subfilters. In the examples of this chapter, we use the general-purpose 
nonlinear-optimization routine fminimax in MATLAB together with the real-rotation theorem, 
see [29], to solve the problem. The real-rotation theorem states that minimizing |f| is 
equivalent to minimizing R{ fe}, VO € [0, 27]. The optimization problem is then solved 
with w and © discretized to dense enough grids. A few hundred and 10-20 points, 
respectively, are typically sufficient in practice. 


2.4.2. Modifications When Using an FRM Filter F(z) 


When F(z) is an FRM filter, we can use essentially the same design steps as outlined above. 
However, a difference is that F(z) is now realized in terms of the two subfilters A(z?) and 
Bo(z) or, equivalently, Fy(z) is now realized in terms of the three subfilters Ag(z"), Boo(z) and 
Boi (z). This means that three parameters, Ñ Ay Ñ B, and L, instead of only one parameter, Ñ F, 
need to be estimated. Given the same approximation error and band edges as before, F(z) 
as well as Ñ Ay Nz, and T, can be obtained as outlined in [7]. It is noted here that the design 
of F(z) in Step 2 now corresponds to a nonconvex problem due to cascaded subfilters in the 
FRM approach. In [7], this is solved via initial linear optimizations and further nonlinear 
optimization, similar to the approach given above for the two-rate based structure. 


6 For a right-band specification, a highpass filter F(z) is designed instead, in the stopband w € [0, we/2]. 
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Figure 2. Magnitude response and approximation error of the two-rate and FRM based filter in Example 1. Throughout this 
chapter, we have used T = 1 for simplicity. 


2.5. Examples 


Consider a first-degree differentiator with the desired function [4] 
D(jw) = eI@ (Not Nr) /4 ig) (17) 


in the frequency region wT € [0, we], O0 < we < 7. This function can be approximated by 
a Type II linear-phase FIR filter, i.e., by a filter of even order and with an anti-symmetric 
impulse response, see [4]. 


Example 1: we = 0.957, and de = 0.01 (—40 dB). Using a conventional differentiator, 
the specification is met by a 60th-order filter which requires 30 multiplications and 59 
additions in an implementation. Using instead the two-rate and FRM based approach, with 
L = 5, we can meet the specification with filter orders 22, 18, and 2, for A(z), Bo(z), and 
G(z), respectively. The corresponding overall realization requires 17 multiplications and 
31 additions. Thus, multiplication and addition savings of 43% and 47%, respectively, are 
achieved. The savings are however dependent of the bandwidth wc as will be illustrated 
below in Example 2. As always when using linear-phase FRM filters, the price to pay is a 
somewhat increased delay, and a few more delay elements. In this example, the delay is 
increased from 30 to 32 samples whereas the number of delay elements is increased from 60 
to 64. The increase is thus only 7%. The overall filter frequency response is plotted in Fig. 2. 


Example 2: Figure 3 shows the number of multiplications required for the conventional 
direct-form filter and the two-rate based filter, both approximating first-degree Type III 
differentiators with approximation errors of 6 = 0.01, 0.001, 0.0001 (—40, —60, —80 dB). 
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As the plots reveal, the complexity savings using the two-rate based filter is increased 
substantially when the bandwidth approaches 7. The break-even point is somewhere 
around we = 0.87 from which the savings increase approximately linearly with increasing 
bandwidth. In the region between 0.87 and 0.987, the savings go from around zero up to 
some 65%. Similar savings are obtained also for the number of additions as it is proportional 
to the number of multiplications. Again, a price to pay for the arithmetic complexity 
reductions is a moderate increase of the delay and number of delay elements, typically 
between some 5% and 20%. 


From the results in [17, 22], the number of multiplications required for a regular Type III 
differentiator can be estimated as 


N 0.810[—log,,(6-)]°7!? 

Mregular z x Sos £ $ (18) 
For the two-rate based differentiators, we have instead from [17] 

p 0.884[— log, (ôe) |0872 

Kwornte = 7 [= l08g10(8c)] (19) 


m — 0.956wc 


Comparing the two expressions, we see that the main difference is the multiplicative constant 
0.956 in front of wc in the latter expression. This explains why the savings increase with 
increasing bandwidth, as illustrated in Fig. 3. 


2.6. Generalization to M > 2 


The two-rate based scheme can readily be extended to the one depicted in Fig. 4(a) where 
the interpolation factor is an arbitrary integer M. Here, the basic principle is thus to first 
interpolate with M via the interpolation filter F(z). Then the actual function is again 
approximated by G(z). Finally, downsampling by M occurs. Using multi-rate theory, one 
finds again that this structure has the LTI system equivalent seen in Fig. 4(b). That is, the 
overall transfer function is 


M 
H(z) = Fo(z)Go(z)+ } z 'Fm(z)GMm-m(2), (20) 


where Fm(z) and Gm(z) are polyphase components of F(z) and G(z) in the polyphase 


representations 
M-1 M-1 
Fa@j= } z™En(zM), G(z) = L 2 ee) (21) 
m=0 m=0 


Using an Mth-band interpolation filter F(z), also the generalized scheme is appropriate for 
left-band and right-band systems. It has turned out though that the case with M = 2 typically 
is the most efficient choice which is why that case has been considered in detail in this section. 
This is because the additional cost of F(z) exceeds the additional savings of G(z) when going 
from M = 2 to M > 2. This in turn is due to the fact that the complexity of G(z) is already 
very low for M = 2. A more detailed discussion on this is found in [17]. 
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Figure 3. Number of multiplications versus bandwidth w.T for the conventional direct-form filter (dashed line) and proposed 
two-rate based filter (solid line), for a first-degree Type Ill differentiator with the approximation errors ôe = 0.01, 0.001, 0.0001. 


Throughout this chapter, we have used T = 1 for simplicity. 
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Figure 4. (a) Two-rate approach with arbitrary M. (b) Equivalent single-rate realization. 


3. Two-rate based structure for mid-band systems 


This section extends the results to mid-band systems which target the region w € [we, Wea, 
0 < wa < we < m. Example applications include fractional-degree differentiators and 
integrators, see [30-33]. For later discussions, we define the don’t-care bands A; and Az as 


Ay = wa, 


Ao = T — We. 


(22) 
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In principle, we can again make use of the scheme in Fig. 4 with a lowpass filter F(z) but it is 
not efficient for mid-band systems. This is because the filter G(z) then needs to approximate 
the desired function in the band between wa /M = A,/M and wa/M = (7 — A2)/M. 
Although this implies that the width of the upper don’t-care band of G(z) is increased 
substantially to roughly (M—1)7/M instead of the original Ay = 7 — we, its lower 
don’t-care band, A;/M = w,,/M, becomes M times narrower. This means that the 
complexity of G(z) may thereby even increase, not decrease. In the left-band case, this is 
not a problem as there is no don’t-care band to the left. 


The width of both the lower and the upper don’t-care bands of G(z) can be increased by 
using a bandpass filter F(z) instead of a lowpass filter. This also means that we have to use 
M > 2. Again, it appears that the most efficient case is for the lowest possible M which is 
here M = 3. The reason for this is two-fold. First, odd values of M makes it possible to 
center the passband of G(z) around 7/2, which maximizes the minimum of its lower and 
upper don’t-care bands. Second, the complexity of F(z) alone reduces with reduced M, in 
accordance with the discussion in [17] for the left-band case. In addition, the use of M = 3 
instead of M > 3, makes it possible to double the amount of sparsity of F(z), and thus its 
efficiency, by expressing it as a periodic filter. 


Here, G(z) is to approximate D(jwM — j(K — 1)7z) in the frequency region w € [tS 2 wl ) 
where 

wl) = (K-1)a/M+41/M, wS) =Kr/M- M/M, (23) 
with K being an appropriately chosen odd integer. For M = 3, one should use K = —1. 


After the downsampling by M, the above region is mapped to the targeted region w € 
[we1, w02]. Further, F(z) is to approximate M in the same region as that of G(z) and zero in 
the corresponding image bands created in the upsampling. Hence, F(z) is here a bandpass 
filter with passband and stopband edges at 


F G F G 
iy Slag a =e (24) 
and 
wl = (K—1)n/M— A/M, w = Kn/M+A2/M, (25) 
respectively. Moreover, with M = 3 and K = —1, F(z) is a symmetric bandpass filter centered 


on 7/2. Consequently, it can be expressed as 
F(z) = 3P(z’) (26) 
where P(z) is a unity-gain-passband third-band highpass filter. The polyphase 
decomposition of P(z) is then P(z) = 1/3 + z~!P,(z3) + z~*P)(z°) which leads to F(z) = 
1 +3z72P (zf) + 3z~4P)(z°) and the polyphase components 
Fo(z) =1, Fy(z) =327'P,(2*), F(z) = 3P, (2). (27) 


A filter F(z) of the form above requires roughly only one third of the complexity of a general 
filter of the same order. 
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Ny Ng Nf DH DE Mult Add 

Regular 124 - - 62 124 125 124 

Two-rate, regular F(z) = P(z*) 126 10 372 63 126 73 134 
Two-rate, FRM F(z) = P(z?) 140 10 412 70 140 41 59 


Table 2. Results of Example 3. 


3.1. Complexity savings 


As opposed to the case of linear-phase overall filters considered in Section 2, we can 
here achieve complexity savings without using additional FRM techniques. The reason is 
two-fold. First, as seen above, F(z) = P(z?) is already sparse. Second, as a mid-band 
system is often a nonlinear-phase system, the filter coefficient are not symmetric. By using 
the two-rate based structure, symmetry can partially be utilized as F(z) is a symmetric filter 
whereas only the low-order G(z) is unsymmetric. As to the sparsity, the degree of sparseness 
can be increased by realizing P(z) as an FRM third-band filter. Details are given in [18]. 


3.2. Examples 


Example 3: Consider the approximation of a fractional-degree differentiator with the desired 
function D(jw) = eJ@(Ne+Nr)/4(ja7)°5 in the frequency band w € (0.0271, 0.987] and for an 
approximation error of ĉe = 0.01. Figure 5 shows the frequency response and approximation 
error of the two-rate based design. The filter has been designed using essentially the same 
three-step procedure described earlier, but after minor appropriate modifications, as detailed 
in [18]. Table 2 gives the results for the conventional direct-form realization and for the 
two-rate based realizations, both with a sparse regular bandpass filter and a sparse FRM 
bandpass filter. The quantity Dy denotes the integer part of the group delay whereas DE 
denotes the number of delay elements. As seen from the table, substantial savings are 
achieved using the two-rate based structures, especially when the FRM technique is also 
utilized. As usual when using the FRM technique, one has to pay a price in a somewhat 
increased delay. It is also noted that the savings increase /decrease with increased / decreased 
bandwidth (decreased/increased width of the don’t-care bands). This is in line with the basic 
two-rate based scheme and it was exemplified earlier in Example 2. 


4. Multi-function systems 


In this section, we will discuss the extension to the realization of multifunction systems. The 
two-rate based approach is even more efficient for such systems as the same F(z), and thus 
the same Fo(z), is shared between all functions. We will illustrate this for Farrow-structure 
based (see [34]) variable fractional-delay (VFD) filters. As an example will reveal, the two-rate 
based structure offers dramatic complexity reductions in this application, even without 
using the additional FRM approach. However, incorporating the FRM approach, further 
complexity savings are obtained. 


4.1. Variable fractional-delay filters 


Variable fractional-delay filters find applications in many different contexts like interpolation, 
resampling, delay estimation, and signal reconstruction, see [35-40]. 
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Figure 5. Fractional-degree differentiator responses in Example 3 using the two-rate based structure with an FRM bandpass 
filter targeting wa = 0.027 and we = 0.9877. 


The VFD filter, with a transfer function H(z,d), is for z = el to approximate the ideal VFD 
filter frequency response 


D(jw,d) = e iw(Dutd) (28) 


where Dy is a fixed delay which usually is an integer or an integer plus a half. Further, d 
is the fractional delay. The ideal response should be approximated in the band w € [0, wc], 
0 < we < 7, and for all fractional delays d € [—0.5, 0.5] meaning that a whole sampling 
period (interval) is covered. In general, the sampling period is T, but we have used T = 1 in 
this chapter for simplicity. 


Using the Farrow structure, H(z,d) is expressed in the form 


H(z,d) = D d H;(z). (29) 
k=0 


where H(z) are fixed subfilters which, essentially, realize the weighted differentiators 
eJ@DH x (—jw)* /k! This follows immediately by truncating the Taylor series expansion 
of eiei, see [41]. Further, when there are no restrictions on the fixed part of the delay, 
it is possible and efficient to use linear-phase subfilters H,(z), thus with symmetric or 
antisymmetric impulse responses. We then have Dy = Ny/2, and the following two 
different cases. When G,(z) are of even order Nc, they are of Type I (Type II) for even 
(odd) values of k. This results in an integer Dy. In the odd-order case, G;(z) are instead of 
Type II (Type IV) for even (odd) values of k. In this case, Dy is an integer plus a half. In both 
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Figure 6. Farrow structure realizing the VFD filter transfer function in (29). 


cases, the impulse responses are symmetric (anti-symmetric) for even (odd) values of k, thus 
gx(n) = (—1)kgi (Ng — n). 


Figure 6 shows the regular Farrow structure realizing (29). As seen, the problem amounts to 
realizing the L + 1 differentiator functions with ideal responses (—jwT)*/k!. In other words, 
it essentially corresponds to the realization of a multi-function system, although, in this case, 
the partial outputs are finally combined via the FD multiplications to form only one output. 
Using now the two-rate based approach introduced in Section 2.2, each H,(z) is realized as 


H,(z) = 2Fo(z)Gyo(z) + 27 PFD 2G, (z) (30) 


where Fo(z) and z—(Dr-1)/2 are again the polyphase components of a linear-phase HB 


interpolation filter F(z) with a passband gain of two and delay Dr, whereas Gyg(z) and 
Gy (z) are the polyphase components of the subfilters G,(z). This follows from sandwiching 
the filter F(z)G(z,d) between the upsampler and downsampler, where G(z,d) approximates 
an FD filter in the region [0, we/2]. That is, 


L 
G(z,d) = 3 d*G,(z). (31) 
=0 


The overall realization is shown in Fig. 7. It is noted that F(z) again can be realized using 
the FRM approach in order to further reduce the complexity, as demonstrated in [19]. In this 
case, F(z) is again realized as in Fig. 1(c) or (d). 


4.2. Design examples 


Example 4: We consider the design of a VFD filter with a bandwidth of we = 0.97. The 
filter has been designed using essentially the same three-step procedure described earlier. 
More details are given in [19]. Tables 3 and 4 summarize the results where the number 
of multiplications and additions covers all fixed subfilters assuming appropriate use of 
direct-form and transposed direct-form realizations. In addition, L general multipliers and 
adders are needed for implementing the FD multiply-and-add chain, but this is required in 
all VFD filter structures. Further, the NRMS and ôq values given in the tables indicate the 
normalized root-mean square error and maximum group-delay error as defined by (33) and 
(36), respectively, in [42], whereas ĉe denotes the maximum of the modulus of the complex 
error. Further, DE denotes the number of delay elements. It is seen from the table that that 
the two-rate based structure is considerably more efficient than the regular Farrow structure. 
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Figure 7. Two-rate based structure realizing the VFD filter transfer function in (29) with H(z) as in (30). 


Linear-Phase L Ngu Ne Ne Na Ng P 
Reg. Farrow, WLS [44] 7 73 n/an/an/a n/an/a 
Hybrid, WLS[42] 7 117 n/a n/a n/a n/an/a 
Simplified [43] 9 73 n/an/an/a n/a n/a 
Two-rate based 7 75 12 138 n/an/an/a 
Two-rate based with FRM 7 87 12 162 46 26 3 


Table 3. Results of Example 4. 


Linear-Phase Dy DE Mult Add NRMS (%) ĉe [dB] Sed 
Reg. Farrow, WLS [44] 36.5 73 191 374 0.00023 -—100.04 0.000446 
Hybrid, WLS[42] 58.5117 148 285 0.00021 —100.21 0.000395 
Simplified [43] 36.5 73 91 158 not reported —102.14* not reported 
Two-rate based 37.5 75 80 149 0.00019  —102.96 0.000318 
Two-rate based with FRM 43.5 87 71 129 0.00019 -—104.15 0.000263 


Table 4. Results of Example 4. *Magnitude and phase delay errors. 


It is also more efficient than two alternative approaches whose results are also included 
in the table, namely for the hybrid structure in [42] and the structure in [43] which meets 
roughly the same specification. It is also seen that the extended structure that utilizes the 
FRM technique offers further complexity reductions. The price to pay in this case is however 
a slight increase of the delay and delay elements, but the figures are still considerably smaller 
than for the structure in [42]. Compared with the regular Farrow structure in [44] and the 
one in [43], one has to pay the moderate price of a delay and delay element increase of some 
3% using the basic structure in Fig. 7(b) and 19% using the extended structure incorporating 
the FRM approach. 


5. Multiple-constant multiplication techniques for the subfilter 
implementations 
This section will discuss implementation details, design trade-offs, and comparisons when 


the multiplications in the filters are implemented using multiple constant multiplication 
(MCM) techniques, which realize a number of constant multiplications using only shifts, 
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Figure 8. Two-rate and FRM based VFD filter in Example 4. Error and group delay responses for 11 evenly distributed values of 
d between —0.5 and 0.5. 


adders and subtracters. The situation is different here though, than for the most commonly 
considered transposed direct-form filter realization, as the proposed structures consist of 
cascaded subfilters. This section will therefore elaborate on these issues and provide design 
examples. The focus here is on VFD filters using the two-rate based structure without the 
additional FRM approach. 


For dedicated hardware implementations, one can take advantage of MCM techniques to 
reduce the implementation cost. Multiplications by constant coefficients can be performed 
using adders, subtracters, and shifts. As adders and subtracters have approximately the same 
implementation complexity we will refer to both as adders. Efficient realization of constant 
multiplications is an active research area and much effort has been focused on the case where 
one input data is multiplied by several constant coefficients. This problem has mainly been 
motivated by single-rate FIR filters, where for a transposed direct form FIR filter the input 
is multiplied by several coefficients, see [45-49]. The resulting implementation of several 
multiplications is denoted multiplier block, as in [45]. 


Work has also been done for sampling rate change with an integer factor in [50] and rational 
factor in [51], where it was shown that FIR filters in parallel can be implemented either using 
one multiplier block or by using a constant matrix multiplication block, as in [52-55], with 
the first approach requiring more delay elements than the latter. As a Farrow filter also is 
composed of several FIR filters in parallel we have the same implementation alternatives 
here, not only the single multiplier block case as reported in [56]. This has been extensively 
discussed in [57]. In Fig. 9, the approach to implement the subfilters proposed in [56] is 
shown. This approach typically requires few additions for the multiplier block. However, a 
separate set of registers is required for each subfilter and the number of structural adders 
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Figure 9. Realization of Farrow filter using transposed direct form subfilters resulting in a multiplier block and several sets of 
registers. 
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Figure 10. Realization of Farrow filter using direct form subfilters resulting in a constant matrix multiplication and a single set 
of registers. 


is high. Alternatively, in Fig. 10, an approach based on the observation in [50] and further 
discussed in [57] is shown. Here, only one set of registers is required and the structural 
adders of the subfilters are merged into the matrix-vector multiplication. 


The Farrow filter part of the two-rate based structure in Fig. 7 can be implemented similarly 
to what is shown in Figs. 9 or 10. For transposed direct form subfilters, as in Fig. 9, 
the corresponding structure would have two inputs, and, hence, result in a constant matrix 
multiplication. Using direct form subfilters, as in Fig. 10, requires two sets of registers, one 
for each input. For the HB filter it is convenient to use a direct form subfilter as the delayed 
input values are easily obtained from the registers. We note that the input to the lower branch 
subfilters in Fig. 7 is just a delayed version of the input, which is available from the upper 
branch subfilter Fy(z). Therefore, it is possible to use the registers of the HB filter as registers 
for direct form subfilters. The resulting structure is illustrated in Fig. 11. 


Naturally, it is also possible to use a transposed direct form HB filter and/or transposed 
direct form subfilters in the Farrow filter part. From a complexity point of view, a transposed 
direct form HB filter will have the same number of adders and registers. However, it will not 
be possible to share registers as shown in Fig. 11. 
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Figure 11. Realization of the filter structure in Fig. 7 using direct form subfilters resulting in a sum-of-products block and a 
constant matrix multiplication. 


5.1. Example and comparisons 


Example 5: We consider a specification where the bandwidth is 0.97 and the modulus of the 
complex error should be below 0.0042. To meet this specification, the Farrow structure in Fig. 
6, with subfilters jointly optimized as outlined in detail in [41], requires 45 fixed multipliers, 
88 fixed adders, and 5 variable multipliers. The two-rate based structure in Fig. 7, with 
subfilters jointly optimized as detailed in [14], requires 30 fixed multipliers, 53 fixed adders, 
and 5 variable multipliers. Thus, in terms of number of multiplications and additions, the 
two-rate based structure is superior. 


To refine the comparison when MCM techniques are applied we must quantize the filter 
coefficients. For a relative comparison, one can use simple rounding. We found that the 
original Farrow structure requires 11 bits to fulfil the requirements whereas the structure in 
Fig. 7 requires 13 fractional bits. The slightly larger number of bits for the two-rate approach 
is explained by the fact that a cascaded filter must meet the requirements which leads to a 
somewhat more stringent requirement on the subfilters, at least when simple rounding is 
used. 


For the regular Farrow structure in Fig. 6, together with the realization in Fig. 9 a total of 
33 adders are required for the multiplier block using the RAG-n algorithm in [45]. This is an 
optimal result since there are 33 different (odd) coefficients as discussed in [58], and, hence, 
there is no need to apply the slightly more efficient algorithms in [47-49]. Furthermore, 80 
structural adders and 118 registers are required for the FIR subfilters. Further, five general 
multipliers and four additional adders are required (for both the regular Farrow and two-rate 
based filters). Alternatively, using the proposed structure in Fig. 10 the constant matrix 
multiplication can be realized using 96 adders using the algorithm in [53]. In addition, 
26 structural adders are required. One observation is that separating the symmetric and 
anti-symmetric subfilters may reduce the complexity as some algorithms work better for 
fewer columns. If this is utilized, the number of adders can be reduced to 107 by applying 
the algorithm in [52] to the resulting two matrices and adding and subtracting the results. 
The number of registers is now decreased to 30, whereas the number of general multipliers 
and additional adders is constant. 


For the two-rate based structure in Fig. 7, the HB filter requires 43 structural adders and 
29 registers. The sum of products are realized by computing the corresponding multiplier 
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Filter structure Mult Add Registers 
Farrow with transposed direct-form subfilters 5 117 118 
Farrow with direct-form subfilters 5 111 30 
Proposed two-rate based structure 5 85 35 


Table 5. Results of Example 5. 


block with RAG-n and transposing it. For the constant matrix multiplication, 38 adders 
are required. This number is not reduced by separating the symmetric and anti-symmetric 
subfilters. A total of six additional registers are required as well as the five general multipliers 
and four additional adders. 


The results are summarized in Table 5. It is seen that the two-rate based structure still has 
the lowest complexity for most implementation technologies as five registers will typically 
be less complex to implement than 26 adders. Furthermore, whereas the use of transposed 
direct form subfilters for the Farrow filter, as proposed in [50, 57], reduces the number of 
adders related to the multiplication, it is for the two-rate approach still more efficient to use 
direct form subfilters. 


6. Conclusion 


This chapter has reviewed recent two-rate based structures and their design for obtaining 
efficient wide-band FIR systems. Left-band, right-band, and mid-band systems, as well as 
single-function and multi-function systems, were covered. Several design examples were 
given, for differentiators and VFD filters (a special case of multi-function systems), revealing 
dramatic complexity savings for wide-band specifications. More details can be found in 
[12-19]. 
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1. Introduction 


On the fixed-point implementation of digital filters, undesirable finite-word-length (FWL) 
effects arise due to the coefficient truncation and arithmetic roundoff. These FWL effects 
must be reduced as small as possible because such effects may cause serious degradation of 
characteristic of digital filters. Ly-sensitivity is one of the evaluation functions which evaluate 
the coefficient quantization effects of state-space digital filters [1-14]. The Lp-sensitivity 
minimization is quite beneficial technique for the synthesis of high-accuracy digital filter 
structures, which achieves quite low-coefficient quantization error. 


To the Lo-sensitivity minimization problem, Yan et al. [1] and Hinamoto et al. [2] proposed 
solutions using iterative calculations. Both of the solutions in [1] and [2] try to solve 
nonlinear equations by successive approximation. Their solutions do not guarantee that the 
Lo-sensitivity surely converges to the minimum Lp-sensitivity since their solutions are not 
analytical solutions. It is necessary to derive some analytical solutions to the Lo-sensitivity 
minimization problem in order to guarantee that their conventional solutions surely derive 
the minimum L»-sensitivity. 


This chapter presents analytical approach for synthesis of the minimum L»-sensitivity 
realizations for state-space digital filters. In Section 3, we derive closed form solutions 
to the L-sensitivity minimization problem for second-order digital filters [12, 13]. This 
problem can be converted into the problem to find the solution to fourth-degree polynomial 
equation of constant coefficients, which can be algebraically solved in closed form. Next, we 
reveal that the Lo-sensitivity minimization problem can be solved analytically for arbitrary 
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(a) Nth-order digital filter (b) Second-order digital filter 


Figure 1. Block diagram of a state-space digital filter. 


filter order if second-order modes are all equal [14] in Section 4. We derive a general 
expression of the transfer function of digital filters with all second-order modes equal. We 
show that the general expression is obtained by a frequency transformation on a first-order 
prototype FIR digital filter. Furthermore, we show the absence of limit cycles of the 
minimum L»-sensitivity realizations [11] in Section 5. The minimum L»-sensitivity realization 
without limit cycles can be synthesized by selecting an appropriate orthogonal matrix in the 
coordinate transformation matrix. 


2. Preliminaries 


This section gives the preliminaries in order to lay groundwork for the main topics of this 
chapter, which appear in later sections. In Subsection 2.1, we begin with introduction 
of state-space digital filters. Subsection 2.2 provides the introduction of L»-sensitivity. 
Subsection 2.3 explains coordinate transformations, the operation for changing the structures 
of state-space digital filters under the transfer function invariant. Subsection 2.4 formulates 
the Lo-sensitivity minimization problem. 


2.1. State-space digital filters 


It is beneficial to introduce the state-space representation for the synthesis of high accuracy 
digital filters. For a given Nth-order transfer function H(z), a state-space digital filter can be 
described by the following state-space equations: 


x(n + 1)=Ax(n) + bu(n) (1) 
y(n) =ex(n) + du(n) 2) 


where x(n) € RN*! is a state variable vector, u(n) € R is a scalar input, y(n) € Risa 
scalar output, and A € RNN, b e RNX1 c e RIXN, d € R are real constant matrices called 
coefficient matrices. The block diagram of the state-space digital filter (A, b, c,d) is shown 
in Fig. 1(a). In case of second-order digital filters, the block diagram in Fig. 1(a) can be 
rewritten as shown in Fig. 1(b). The transfer function H(z) is described in terms of the 
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coefficient matrices (A,b,c,d) as 
H(z) = c(zI — A)~!b +d. (3) 


In this chapter, the state-space representation (A,b,c,d) is assumed to be a minimal 
realization of H(z), that is, the state-space representation (A,b,c,d) is controllable and 
observable. The transfer function H(z) in Eq. (3) can be rewritten as 


adj(zI — A) 


B= det(zI — A) 


b+d. (4) 


We know from the above equation that the poles of H(z) are the solutions of the characteristic 
equation det(zI — A) = 0, that is, the eigenvalues of the coefficient matrix A. Since we 
assume that the transfer function H(z) is stable, all absolute values of the eigenvalues of the 
coefficient matrix A are less than unity. It follows from Eq. (4) that the absolute values of 
poles of H(z) are less than unity. 


2.2. Lo-sensitivity 


The L»-sensitivity is one of the measurements which evaluate coefficient quantization errors 
of digital filters. The L2-sensitivity of the filter H(z) with respect to the realization (A, b, c,d) 
is defined by 


: 2 
N 27 (e 
1 dH(e!”) 
S(A, b,c) = f dw 
ee ban o0 | a 
F 2 k 2 
N Qn jw N 27m jw 
reef dH(e!”) othe f dH(el)\ 
14 27 Jo Ob; = 27 Jo oc] 
_||AH(z) |? aH) aae) 6) 
dA |a || ob la || oc İz 


where || - ||2 denotes the Ly-norm. The derivatives of the transfer function H(z) with respect 
to the coefficient matrices are described by 


əƏH(z) _ əƏH(z) _ ƏH(z) _ 
A = & (ET), Se = GT), SH = FT) (6) 
where F(z) and G(z) are defined by 
F(z) = (zI—A)~1'b (7) 


G(z) = e(zI— A)! (8) 
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respectively. Substituting Eqs. (6) into Eq. (5), the Ly-sensitivity can be rewritten as 


z) ||? z) ||? z) |2 
sanoe PL Peel 
=e (2F |, -+ eok + FTI. 


We can express the L»-sensitivity S(A, b,c) by using complex integral as [1] 


S(A,b,c)=tr [z f rocoreee?] 


Z 
d d 


Applying Parseval’s relation to Eq. (10), Hinamoto et al. expressed the L-sensitivity in terms 
of the general Gramians such as [2] 


S(A,b,c) = tr(Wo)tr(Ko) + tr(Wo) + tr(Ko) 4 2)" te(W))tr(K;). (11) 
i=1 


The general controllability Gramian K; and the general observability Gramian W; in Eq. (11) 
are defined as the solutions to the following Lyapunov equations: 


K;=AK;A! + ; (A'bb" j bb” (AT) (12) 
W;=ATW;A + ; (cTca! F (AT)icTe) (13) 


for i = 0,1,2,---, respectively. The general controllability and observability Gramians are 
natural expansions of the controllability and observability Gramians, respectively. Letting 
i = 0 in Eqs. (12) and (13), we have the Lyapunov equations for the controllability Gramian 
Ko and the observability Gramian Wọ as follows: 


Ko =AK0A! + bb? (14) 
Wo=A WoA+ cle. (15) 


The controllability Gramian Kg and the observability Gramian Wọ are positive definite 
symmetric, and the eigenvalues 0? (i =1,--- ,N) of the matrix product KgW 9 are all positive. 
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Second-order modes are defined by the square roots of the eigenvalues 6;’s as follows [3, 5]: 


(01,--+ ,On) = \/Eigenvalues of KoWo. (16) 


In the field of digital signal processing, the controllability and observability Gramians are 
also called the covariance and noise matrices of the filter (A, b, c,d), respectively. 


2.3. Coordinate transformations 


It is well known that the number of state-space realizations of a transfer function is infinite 
since choice of the state variable vector x(n) is not unique. We can change the filter structures 
of state-space digital filters by the operation called coordinate transformation under the transfer 
function invariant. 


Let T be a nonsingular N x N real matrix. If a coordinate transformation defined by 
x(n) = T~!x(n) (17) 


is applied to a filter structure (A,b,c,d), we obtain a new filter structure which has the 
following coefficient matrices: 


(A,b,¢,d) = (T-!AT,T~'b,cT, d). (18) 


It should be noted that the coordinate transformation does not affect the transfer function 
H(z), that is, 


H(z) =e(zI — A)~'b+d 
=c(zI — A)~'b +d 
=H(z). (19) 


It implies that there exist infinite filter structures for a given transfer function H(z) since 
nonsingular N x N matrices exist infinitely. Therefore, one can synthesize infinite filter 
structures by the coordinate transformation with keeping the transfer function invariant. 


Under the coordinate transformation by the nonsingular matrix T, the general controllability 
Gramian K; and the general observability Gramian W; are transformed into K; and W; given 


by 
(K;,W;) = (T~'K;T~", TTW;T) (20) 


respectively. Letting i = 0 in Eqs. (20) yields 


217 


218 Digital Filters and Signal Processing 


(Ko, Wo) = (T KoT", T'WoT) (21) 
From Eggs. (21), we have 
KoWo = T-!KoWoT (22) 


which shows that KoWo has the same eigenvalues of KoWoọ. Thus, the second order modes 
defined by Eq. (16) are invariant under the coordinate transformations. It implies that 
second-order modes depends on only the transfer function. 


2.4. Lo-sensitivity minimization problem 


The value of L-sensitivity depends on not only the transfer function H(z) but also the 
coordinate transformation matrix T. The L2-sensitivity of the filter (T-!AT, T~1b,cT, d) can 
be expressed in terms of the complex integral as 


S(T ‘AT,T ‘b,cT) 


1 Tat 1 -1 tiy p-TiZ 
+r [fT G ecer] +tr laa faa? FETT] o3 


or in terms of the general Gramians as 
S(T 'AT,T~‘b,cT) 
=tr(T’WoT)tr(T~'KoT~') + tr(T’ WoT) + tr(T KoT") 


+2) tr(T’W,T)tr(T~'K;T~*). (24) 
i=1 


i= 


The L»-sensitivity (23) can be expressed as the function of the positive definite symmetric 
matrix P as follows [1]: 


+tr E faa cte)ee) P] +tr E faa FG)F'G) P Ë E) 


where P = TT’. Similarly, the L2-sensitivity (24) can be expressed as the function of the 
positive definite symmetric matrix P as follows [2]: 
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S(P)=tr(WoP)tr(KoP +) + tr(WoP) + tr(KoP +) 


+2 y tr(W;P)tr(K;P~!) (26) 
i=1 


where P = TT’. The problem we consider here is to derive the optimal positive definite 
symmetric matrix Popt, which gives the global minimum of S(P) as follows: 


a =o. 27) 
oP P=P y 


If one can obtain the optimal positive definite symmetric matrix Popt, the optimal coordinate 
transformation matrix Topt is given by 


1 
Topt = Pèp U (28) 


where U is an arbitrary orthogonal matrix. It implies that the minimum L3-sensitivity 
realizations exist infinitely for a given digital filter H(z). The minimum L3-sensitivity 
realizations have freedom for orthogonal transformations. 


3. Analytical solutions to the L2-sensitivity minimization problem for 
second-order digital filters 


This section proposes analytical synthesis of the minimum L»-sensitivity realizations for 
second-order digital filters. We propose closed form solutions to the Lp-sensitivity 
minimization problem of second-order state-space digital filters. The proposed closed 
form solutions can greatly save the computation time and guarantee that the Lo-sensitivity 
obtained by the iterative algorithm of the conventional method surely converges to the 
theoretical minimum. We show that the L»-sensitivity is expressed by a simple linear 
combination of exponential functions, and we can obtain the minimum Lp-sensitivity 
realization by solving a fourth degree polynomial equation of constant coefficients in closed 
form without iterative calculations [12, 13]. 


3.1. Problem formulation 


We adopt the balanced realization (Aj, bp, Cp, dp) as the initial realization to synthesize the 
minimum L»-sensitivity realization. The coefficient matrices (Ap, bp, Cp, dp ) satisfy symmetric 
properties as follows: 


Al = EA, cl = Xb, (29) 
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where & is a signature matrix defined as follows: 


X = diag(o1,--- ,7n), 0, = +1 (i=1,--- ,N). (30) 


We exploit the symmetric properties of the balanced realization in order to simplify 
the Lo-sensitivity formulation and minimization in the following discussion. Under this 
condition, the Ly-sensitivity S(P) in Eq. (26) is rewritten as 


S(P)=tr(WP P)tr(KP P~!) + tr(WP P) + tr(KP P=) 


+2 Y tr(W) Pytr(K 
i=1 


1 


Pp) (31) 


and thus, the L2-sensitivity minimization problem is formulated as follows: 
min S(P) in Eq. (31) (32) 
where P is an arbitrary positive definite symmetric matrix. 


We derive the optimal positive definite symmetric matrix Popt which gives the global 
minimum of the L9-sensitivity S(P) in Eq. (31). 
3.2. Second-order digital filters 


Consider second-order digital filters with complex conjugate poles given by 


a a* 
H(z) = oar (33) 


where (A,A*) are complex conjugate poles, « is a complex scalar, and d is a real scalar 1. We 
define scalar parameters P, Q, and R as follows: 


—_ lal 
1- |A]? 
a 


R+jQ= 5 


(34) 


(35) 


which can be calculated directly from the transfer function H(z). The closed form expression 
of the balanced realization of the filter H(z) is given as follows [15]: 


K— K7! K+ K7! 
Àr Ài Ài [Hit pe 
Ap |bp 25 2 
= K+K K= Ko} (36) 
Cp |dp 5 Ai Ard 5 Ail — p2 
p1 + p2 -(m-m) | d 


1 For also second-order digital filters, we can derive analytical solutions to the Lz-sensitivity minimization problem 
[13]. 
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where 


; y P 
À = Àr + JÀi, & = Oy + jaj, K = VS 
37 
_ a mt oF 
pı 7 , M2 K sign(ar). 


Using the parameters P,Q, and R, the controllability Gramian KP and the observability 


Gramian wh of the balanced realization (Ap, bp, cp, dp) can be expressed as follows: 


) =wy” =e (38) 
© =diag(6, 02) 


=diag(,/P2 — Q2 + R, \/P2 — Q2 — R). (39) 


3.3. Property of the positive definite symmetric matrix P 


In this subsection, we consider the property of the positive definite symmetric matrix P. The 
following two theorems lead a symmetric property of the optimal positive definite symmetric 
matrix Popt [1]. 


Theorem 1. [9] Lo-sensitivity S(P) has the unique global minimum, which is achieved by a positive 
definite symmetric matrix Popt satisfying 


aS(P) 


=0. (40) 
oP — 


Theorem 2. [1] Ifa positive definite symmetric matrix Popt satisfies 


0S(P 
ee (41) 
P P=P or 
then the positive definite symmetric matrix EP oe also satisfies 
0S(P 
> =0 (42) 
oP |P-EP}E 


for the signature matrix & which satisfies Eq. (29). 
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The derivative 0S(P)/dP is given by differentiating S(P) in Eq. (31) with respect to P as 


cele =(1+ (KPP HWP 425 tr(K) py w! 
i=1 


b) 
pot (a + tr(WP)) KM +27 cw 29K.) po, (43) 
i=1 


From Theorem 1 and Theorem 2, it is proved that the optimal positive definite symmetric 
matrix Popt has the following symmetric property [1]: 


Popt = PoE (44) 


for the signature matrix & which satisfies Eq. (29). We will thus search the optimal positive 
definite symmetric matrix Pop: among the positive definite symmetric matrices P which 
satisfy 


P =P. (45) 


3.4. Closed form expression of the positive definite symmetric matrix P 


In this subsection, we consider the case of second-order digital filters and give the closed 
form expression of the positive definite symmetric matrix P. When we restrict ourselves to 
the second-order case of state-space digital filters, we can give an closed form expression of 
the positive definite symmetric matrix P which satisfies Eq. (45), considering the form of the 
signature matrix Ł classified into the following two cases: 


ÆI 


. = +diag(1,1) (46) 


x= +diag(1, —1). 


For each case, we next consider the closed form expression of the positive definite symmetric 
matrix P which satisfies Eq. (45). 


In case of & = +I, Eq. (44) yields Popt = Pig that is, Popt = I. It means that the minimum 
Lo-sensitivity realization can be synthesized without any coordinate transformation to 
the balanced realization, that is, the initial realization. In other words, the minimum 


Lo-sensitivity realization is the balanced realization as follows: 


(Aopt, bopt, Copt, dopt) = (Ap, bp, Cp, dp). (47) 


Thus, we need no more discussion on this case since the minimum Lp-sensitivity realization 
is already achieved as the balanced realization. 
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On the other hand, in case of £ = +diag(1, —1), the authors have derived the closed form 
expression of a positive definite symmetric matrix P which satisfies Eq. (45) as follows: 


cosh(p) sinh(p) 


P= | sinh(p) cosh(p) 


(48) 


where p is a real scalar variable [12, 13]. 


3.5. Closed form expression of the Lz-sensitivity S(P) 


In this subsection, the closed form expression of the L-sensitivity of second-order digital 
filters is given. We give the closed form expression of the Ly-sensitivity S(P) in Eq. (31). We 
first express the general Gramians of the balanced realization (Ap, bp, cp, dp) as follows: 


1 
K{") =; (AbKy” + Ky” (45)') 
-17 Tyi 
=5 (4,0 + @(45)') (49) 
1 ; : 
WI?) =5 (Wy At + (api we) 
_ 1 i T\i 
=5 (oai, + (A1) a) (50) 


We express the Ly-sensitivity S(P) by substituting Eqs. (49) and (50) into Eq. (31) as follows: 


S(P) = tr(@P)tr(@P~') + tr(@P) + tr(@P~') +2 3 tr(©AŻP)tr(Ə(AT)İPT!). (51) 
i=1 


The L»-sensitivity S(P) in Eq. (51) can be expressed more simply. Exploiting the symmetric 
property of coefficient matrix A, and P given in Eqs. (29) and (45) respectively, we can 
rewrite the Ly-sensitivity S(P) as 


S(P) =2tr(@P) — (tr(@P))* +2 3 (tr(@ajp))” (52) 
i=0 


In order to give the closed form expression of the L»-sensitivity S(P) in Eq. (52), it is 
necessary to derive the closed form expressions of matrices Ap, ©, and P. The closed form 
expressions of matrices A, and @ are given in Eqs. (36) and (39), respectively. The closed 
form expression of the positive definite symmetric matrix P is given in Eq. (48). Substituting 
the closed form expressions of matrices Ap, O, and P into Eq. (52) gives the closed form 
expression of the Lo-sensitivity S(p) as 


S(P) = S(p) = D Spe”P, (53) 
n=—2 
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It is remarkable that Eq. (53) is a simple linear combination of exponential functions which 
does not contain infinite summations. These coefficients s,’s are easily computed directly 
from the transfer function H(z) [12, 13]. 


3.6. Synthesis of minimum L»-sensitivity realizations 


The parameter p which minimizes S(p) in Eq. (53) can be derived by solving the following 
equation with respect to p: 


2 
oy) = D nsye"P = 0. (54) 
Letting 6 = e? gives 


X nsnB" = 0. (55) 


The above equation is a fourth-degree polynomial equation with respect to 6 of constant 
coefficients. In 1545, G. Cardano states in his book entitled Ars Magna (its translated edition 
is [16]) that there exists the formula of solutions for fourth-degree polynomial equations. 
Therefore, Eq. (55) can be solved analytically. Eq. (55) has four solutions, from which the 
positive real solution Bopt = ePort is adopted to derive the optimal positive definite symmetric 
matrix Popt as 


sinh(popt) cosh(popt) 


1 
Bopt + Pop Bopt — Popi 


Bopt — Bopt Bopt + Bol 


w=] cosh( a. sinh(Popt) | 
1 
=3 (56) 


The diagonalization of the optimal positive definite symmetric matrix Popt = ToptT opt is 


given by 
ZA ihe Bopt 0 i 1 
fe) 
v2 v2 opt v2 V2 
=R" BoptR. (57) 


Once the optimal positive definite symmetric matrix Popt is derived, the optimal coordinate 
transformation matrix Topt is calculated as 


1 
Topt = P2,,U (58) 
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where U is an arbitrary orthogonal matrix. We have to note that the optimal coordinate 

transformation matrix Topt is not unique because of the non-uniqueness of matrix U. By 

letting U = I in Eq. (58), for instance, one of the optimal coordinate transformation matrices 
_ pl/2 +. gi 

Topt = Poot is given by 


1 


Topt =P pr 
1 
=R" BR 
1 1]fg? o i oi 
v2 v2 0 Bo] L~ v2 V2 


2 at 1 A 
= 1 Bopt F Bopt Bopt z Bopt 
5 1 1 1 1 


z | ght opt Popt Fon (59) 
Bopt = Bopt Bopt F Popt 


We can give a geometrical interpretation of the above optimal coordinate transformation 
matrix Topt. In Eq. (59), R is an orthogonal matrix, which means 7/4[rad] rotation of 
the coordinate axes, Bopt is a positive definite diagonal matrix, which means a simple 
scaling of each coordinate axis. Eq. (59) shows that the minimum Lp-sensitivity realization 
is obtained by the following operations for the coordinate axes of the initial balanced 
realization; —7t/4[rad] rotation, simple scaling, and +7¢/4[rad] rotation. Finally, the 
minimum Lp-sensitivity realization (Aopt, bopt, Copt, dopt) is synthesized as follows: 


| Aopt | bopt | — (60) 


Copt |dopt 


Toptâb Topt | Topi by 
Th | 


and the minimum L»-sensitivity Smin, which is achieved by popt = log(opt), is expressed by 
substituting Popt into Eq. (53) as 


2 


Smin = D SnBopt- (61) 
n=2 


3.7. Numerical examples 


We present numerical examples to demonstrate the validity of the proposed method. 
Consider a second-order digital filter H(z) with complex conjugate poles, of which transfer 
function is given by 


ny a* 
se | z—X* Ba 
_ 0.0396 + 0.0793z~! + 0.039627? 


1 — 1.3315z71 + 0.49z-2 


(62) 
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Figure 2. Frequency responses and zero-pole configurations of H(z). 


where A = 0.7 exp(j0.172), « = 1.6657 — j6.3055, and d = 1. The frequency responses and 
zero-pole configurations of digital filter H(z) are shown in Fig. 2. From the transfer function 
H(z), parameters P, Q, and R are calculated as 


P = 0.5068, Q = —0.2947, R = 0.25. (63) 
The coefficients s„’s are computed as follows: 
(S_2,8—1,80,51,82) = (0.3345, 0.8246, 0.8987, 0.8246, 0.7951). (64) 


We solve the following fourth-degree polynomial equation to derive the optimal solution 
Bopt: 


2 


D nsnp” = 0. (65) 


n=—2 


The fourth-degree polynomial equation above has the following four solutions: 


p = 0.8568, —0.6960, —0.3396 + j0.7682. (66) 


We adopt Bopt = 0.8568, which is a positive real scalar, to derive the optimal positive 
definite symmetric matrix Popt. We can derive the minimum L>-sensitivity realization 
(Aopt, bopt, Copt, dopt) in closed form as follows: 


Acct 0.7810 0.2451|0.4751 
Eas = | —0.2451 0.5505|0.3061 (67) 
opt opt 0.4751 —0.3061|0.0396 
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Figure 3. Minimum L»2-sensitivity and convergence behaviors of L-sensitivity in conventional methods [1] and [2]. 


of which Lp-sensitivity Smin is 


2 
Smin = È, SnB3pt = 3.6070. (68) 


n=—2 


Figure 3 shows the comparison of our proposed method with the iterative methods reported 
in [1] and [2], where the initial realization is the balanced realization. Our proposed method 
achieves the minimum Lp-sensitivity by only solving a fourth-degree polynomial equation 
without iterative calculations, while both of the methods in [1] and [2] require many iterative 
calculations to achieve the minimum L»-sensitivity. Furthermore, our proposed method can 
guarantee that the Lo-sensitivity surely converges to the theoretical minimum when using 
conventional methods in [1] and [2]. 


4. Analytical solutions to the Lj-sensitivity minimization problem for 
digital filters with all second-order modes equal 


This section reveals that the Lo-sensitivity minimization problem can be solved analytically 
if second-order modes are all equal. Furthermore, we clarify the general expression of the 
transfer functions of digital filters with all second-order modes equal [14]. 


4.1. Analytical synthesis of the minimum L»-sensitivity realizations 


We have discovered that there exist some digital filters whose minimum L»-sensitivity 
realization is equal to the balanced realization. Such digital filters satisfy a sufficient 
condition summarized in the following theorem: 


Theorem 3. Ifall the second-order modes 0;(i = 1,--- , N) of a digital filter H(z) are equal, then 


(Aopt, bopt, Copt, dopt) = (Ap, bp, Cb, dp) (69) 


that is, the minimum Lp-sensitivity realization is equal to the balanced realization. 
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Proof: The general Gramians of the balanced realization are given by Eqs. (49) and (50), 
respectively. If all the second-order modes 6;(i = 1,--- , N) satisfies 
6; = 0 (i=1,-:-,N) (70) 
the controllability and observability Gramians are expressed as 
KP =w = diag(@,--- ,6) = 61. (71) 


Substituting Eq. (71) into Eqs. (49) and (50), the general Gramians are given by 


= 5O(Ah + (AT)Ż). (72) 


b) 


We can express the general Gramians as KP = w! = ©, which is defined by 


Ə; = 504) + (AD) @=0,1,---). (73) 


Substituting K®? = w = @; into Eq. (43) yields 
gK; q. (43) y 


1 


[e0] 
P) (4 + tr(@0oP7!))@o +2 $ tr(0;P7')O; 
i=1 
foe) 
—p-} (a + tr(@gP))@o9 +2 Èire) P. (74) 
i=1 
It is obvious that 
as(P 
iC) o (75) 
oP | py 


which means that the minimum Lp-sensitivity realization can be synthesized without 
any coordinate transformation to the balanced realization, that is, the initial realization. 
Therefore, it is proved that the minimum Lp-sensitivity is equal to the balanced realization. 


4.2. Class of digital filters with all second-order modes equal 


In the previous subsection, we revealed that the L2-sensitivity minimization problem can be 
solved analytically if second-order modes are all equal. We next clarify the class of digital 
filters with all second-order modes equal. We have newly derived a general expression of 
the transfer function of Nth-order digital filters with all second-order modes equal. 
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4.2.1. General expression 


In Ref. [14], we have newly derived a general expression of the transfer function of Nth-order 
digital filters with all second-order modes equal. 


Corollary 1. Let the second-order modes of an Nth-order digital filter H(z) be 0;(i = 1,--- ,N). 
The second-order modes of the transfer function 0H(z) are given by |0|0;(i = 1,--- , N), where 0 is 
a nonzero real scalar. 


Theorem 4. The transfer function of an Nth-order digital filter H(z) with all second-order modes 
equal such as 


6, = 0) = -+ = Oy_-1 = ON (76) 


can be expressed as the following form: 
H(z) = @Hap(z) + p (77) 


where 0 is a nonzero real scalar, p is a real scalar, and Hap(z) is an Nth-order all-pass digital filter. 
The second-order modes of the digital filter H(z) are given by 


6; = |6| (i =1,---,N). (78) 


4.2.2. Frequency transformation 


Furthermore, it is remarkable that the transfer function H(z) in Eq. (77) is generally obtained 
by the frequency transformation on a first-order FIR prototype filter using an Nth-order 
all-pass digital filter. 


Remark 1. An Nth-order digital filter H(z) in Eq. (77) is obtained by the frequency transformation 
such as 


H(z) = Hp(2Z)|21~- Hyp(z) (79) 


where Hp(z) = 6z~! + p is a first-order prototype FIR digital filter and Hap(z) is an Nth-order 
all-pass digital filter. 


The variable substitution in Eq. (79) represents the frequency transformation, where Hp(z) 
is the prototype digital filter. Block diagrams of the prototype digital filter Hp(z) and 
transformed digital filter H(z) are shown in Fig. 4. These figures show that a block diagram 
of the transformed digital filter H(z) can be obtained by simple substitution of the all-pass 
digital filter Hap(z) into the unit delay z-!. Theorem 4 and Remark 1 give us the class of 
digital filters with all second-order modes equal. 
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(a) First-order prototype digital filter (b) Nth-order transformed digital filter 
Hp(z) A(z) 


Figure 4. Block diagrams of prototype digital filter Hp(z) and transformed digital filter H(z). 


4.3. Examples of digital filters with all second-order modes equal 

There are many types of digital filters with all second-order modes equal. We can design 
various digital filters by setting variables 0, p, and Hap(z). 

4.3.1. The unit delay 

The unit delay is the simplest example for digital filter with all second-order modes equal. It 
is obvious that letting 0 = 1, p = 0, and Hap(z) = z~! in Eq. (77) yields the unit delay z~!. 
4.3.2. First-order digital filters 


Any first-order digital filter can be expressed in the form of Eq. (77). Consider a first-order 
IIR digital filter given by 


(80) 


where bo and b; are numerator coefficients, a, is a denominator coefficient. One can easily 
show that Eq. (80) can be rewritten as the form of Eq. (77) where 


ay + z`! 
, Hap(z) = + (81) 


~T+4+ayz7-) 


_ bı — a1 bo _ bo — a,b, 
1—at f 1—at 


(J 


4.3.3. All-pass digital filters 

It is obvious that all-pass digital filters are included in the class of digital filters expressed as 
Eq. (77). The transfer function H(z) in Eq. (77) is an all-pass digital filter when we let 0 = 1 
and p = 0. 

4.3.4. Multi-notch comb digital filters 


The transfer function of an Nth-order multi-notch comb digital filter is given by 
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lta 1l—zN 


Hn (Z) = z i xN (82) 


This filter has N notches at the frequency 27tk/ N[rad] for k = 1,--- ,N. One can easily show 
that Eq. (82) can be rewritten as the form of Eq. (77) where 


1 a—z 


1 
is wee, H ee ee 
57 = gt HA) ae 


(83) 
4.4. Numerical examples 

This subsection gives numerical examples of synthesis of the minimum Lp-sensitivity 
realizations for various types of digital filters with all second-order modes equal. 

4.4.1. First-order FIR digital filters 

Consider a first-order FIR digital filter Hprr(z) given by 


Hpe(z) = 0.540.527! (84) 


of which frequency magnitude and phase responses are shown in Fig. 5 (a). The second-order 
mode of the digital filter Hpyr(z) is 0 = 0.5. The balanced realization (Ap, bp, cp, dp), which 
is equal to the minimum L»-sensitivity realization, of Hpyy(z) is derived as 


faae] = [aoa 


Ch [dp 0.7071| 0.5 
(85) 
and controllability Gramian KO and observability Gramians we are calculated as 
b b 
KP =w® =0.5. (86) 
4.4.2. First-order IIR digital filters 
Consider a first-order IIR digital filter Hyr(z) given by 
0.25 + 0.25271 
H = SS 7 
mR (Z) 105z] (87) 


of which frequency magnitude and phase responses are shown in Fig. 5 (b). The 
second-order mode of the digital filter Hyp(z) is © = 0.5. The balanced realization 
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(d) fourth-order multi-notch 
comb digital filter Hmn (Zz) 


Figure 5. Frequency magnitude and phase responses of digital filters with all second-order modes equal. 


(Ap, bp, Cp, dp), which is equal to the minimum L3-sensitivity realization, of Hyg (z) is derived 


as 


ee = | 


0.5 |0.6124 
0.6124] 0.25 


(88) 


and controllability Gramian KP) and observability Gramians we are calculated as 


0 


= 0.5. (89) 
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4.4.3. All-pass digital filters 
Consider a fourth-order all-pass digital filter Hap(z) given by 


0.5184 — 1.9805z—! + 3.3350z—2 — 2.7507z~3 + z-4 


H = 
ar(z) = 1275072 14333502 2 — 1980523 + 051847 7 00) 
of which poles Ap(p = 1,2,3,4) are given by 
Ay = 0.9 exp(j0.27r), Az = 0.9 exp(—j0.277), (01) 
Az = 0.8exp(j0.277), Aq = 0.8 exp(—j0.277) 


and of which frequency magnitude and phase responses are shown in Fig. 5 (c). The 
second-order modes 6; (i = 1,2,3,4) of the all-pass digital filter Hap(z) are given by 


(01,02, 03,04) = (1,1,1,1). (92) 


The balanced realization (Ap, bp, Cp, dp), which is equal to the minimum L»-sensitivity 
realization, of Hap(z) is derived as 


0.8144 —0.1106 0.2499 —0.5039 | —0.0903 
Alb 0.1599 0.4698 —0.7114 —0.1105|—0.4853 
ag = | —0.1616 0.2997 0.6211 0.1062|—0.6978 (93) 
Cb |%b 0.5318 0.0448 0.0006 0.8453) 0.0252 

| 0.0481 0.8217 0.2137 —0.0894| 0.5184 


and controllability Gramian K®) and observability Gramians we are calculated as 


KP) — w®) = diag(1,1,1,1). ve 


4.4.4. Multi-notch comb digital filters 


Consider a fourth-order multi-notch comb digital filter Hmn (z) given by 


_ 0.9073 — 0.9073z~4 


H 
MN(2) = T osz a) 
of which poles Ap(p = 1,2,3,4) are given by 
À = 0.29, À = =e (96) 
Ag = j0.95, A4 = —j0.95 
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and of which frequency magnitude and phase responses are shown in Fig. 5 (d). The 
second-order modes 0; (i = 1,2,3,4) of the multi-notch comb digital filter Hmn (z) are given 


by 
(01, 02,03, 04) = (0.5,0.5,0.5, 0.5). (97) 
The balanced realization (Ap, bp,cp,dp), which is equal to the minimum L»-sensitivity 


realization, of Hmn (Zz) is derived as 


100 0 0 
re] = 010 0 0 (98) 
Cb |4b 001 0 0 


0 0 0 —0.4102]0.9073 


l 00 0.8145 |0.4102 


and controllability Gramian KP and observability Gramians wi) are calculated as 


KP = wl) = diag(0.5,0.5,0.5,0.5). (99) 


5. Absence of limit cycles in the minimum L>-sensitivity realizations 


This section proves the absence of limit cycles of the minimum L»-sensitivity realization 
from the viewpoint of the controllability and observability Gramians. The minimum 
Lo-sensitivity realizations have freedom for orthogonal transformations. In other words, 
minimum Lp-sensitivity realizations are not unique. We select the minimum Lp-sensitivity 
realization without limit cycles among these minimum Lp-sensitivity realizations. The 
controllability and observability Gramians of the selected minimum L»-sensitivity realization 
satisfy a sufficient condition for the absence of limit cycles [11]. 


5.1. Theoretical proof of the absence of limit cycles 


For high-order digital filters, we synthesize the minimum L»-sensitivity realization by the 
successive approximation methods in [1] or [2], for examples. For second-order digital 
filters, we can synthesize the minimum Lp-sensitivity realization by the closed form solutions 
proposed in Section 3. For both cases, we can construct the minimum L»-sensitivity 
realization without limit cycles. 


We begin by reviewing the procedure to synthesize the minimum L»-sensitivity. We solve the 
Lo-sensitivity minimization problem in (32) adopting the balanced realization (Ap, bp, cp, dp) 
as an initial realization. We obtain the optimal positive definite symmetric matrix Pop. In 
case of high-order digital filters, we can derive the optimal positive definite symmetric matrix 
Popt by successive approximation method in [1] or [2], for example. In case of second-order 
digital filters, we can derive the optimal positive definite symmetric matrix Popt analytically 
as proposed in Section 3. 
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We can give the diagonalization of the matrix Popt as follows: 
Popt = R” BoptR. (100) 
Since the matrix Popt is positive definite symmetric, it can be diagonalized by an orthogonal 


matrix R, and Bopt is a positive definite diagonal matrix. The optimal coordinate 
transformation matrix Topt is given by 


1 
Topt =P opt U 
—R™B?,,RU (101) 
= opt ý 
In the above expression, U is an arbitrary orthogonal matrix. It means that the minimum 
Lo-sensitivity realizations exist infinitely for a given digital filter H(z). The minimum 
Lo-sensitivity realizations have freedom for orthogonal transformations. We show that the 


minimum L»-sensitivity realization does not generate limit cycles if we specify the orthogonal 
matrix as U = RT, which yields 


os 1 
Ta R' Boot: (102) 


Theorem 5. The minimum Lp-sensitivity realization (Aopt, bopt, Čopt dopt), obtained by the 
coordinate transformation by opt such as 


$ ~ a x eae | m ai m 
(Aopt, bopt, Copt, dopt) = (Toptâb Topt, Toptbp, Ch Topt, dp) (103) 


does not generate limit cycles. 


Proof: Under the coordinate transformation by Topt in Eq. (102), the controllability Gramian 
Kr?) and the observability Gramian werd) of the minimum L>-sensitivity realization 


(Aopt, Bopt, Gopt dopt) are expressed as 


KP) oT KT 

= By 4ROR"B, 4 (104) 
WE — Th We) Toot 

=B} ,ROR'B?,, (105) 


where © = diag(61,--- ,6y). From Eqs. (104) and (105), we can derive the relation between 
the controllability and observability Gramians as follows: 
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Balanced realization 


Aopt bopt 
Copt dopt 1 


Minimum Lə-sensitivity realizations without Lə-scaling constraints 


Figure 6. Synthesis of the minimum L2-sensitivity realization which does not generate limit cycles. 


WP) = Bo KOP? Bont. (106) 


Eq. (106) is equivalent to a sufficient condition for the absence of limit cycles proposed in 
Ref. [6]. Therefore, the minimum Lp-sensitivity realization (Aopt, bopt- Copt, dopt) does not 
generate limit cycles. 
Theorem 5 shows that we can synthesize the minimum L»-sensitivity realization without 
limit cycles by choosing appropriate orthogonal matrix U. Fig. 6 shows the synthesis 
procedure of the minimum Lp-sensitivity realization which does not generate limit cycles. 
The coefficient matrices of the minimum Lp-sensitivity realization without limit cycles 
(Aopt, bopt, Zopt, dopt) are given by 


1 


1 1 
3 7 =i Tp! 1 
Eira Z Bat BAR Bopt BopiRbp 


Copt [dopt cpR™B?,. | d 


(107) 


5.2. Numerical examples 


We present numerical examples to demonstrate the validity of our proposed method. We 
synthesize the minimum Lp-sensitivity realizations of second-order and fourth-order digital 
filters which do not generate limit cycles. 
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Figure 7. Frequency Response of digital filter H(z) in Eq. (108). 
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Figure 8. Zero-input responses of H(z) in Eq. (108). 


5.2.1. Second-order digital filters 


Consider a second-order narrow-band band-pass digital filter H(z) given by 


0.0316 + 0.0602z~! + 0.031627? 


H 
(2) 1 — 1.4562z-1 +0.81z 2 


(108) 


The poles of the transfer function H(z) in Eq. (108) are 0.9 exp(4 


+ j0.27t), which are very close 


to the unit circle. The frequency response of the digital filter H(z) in Eq. (108) is shown in 
Fig. 7. The coefficient matrices of the minimum L»-sensitivity realization which is free of 


limit cycle is derived by Eq. (107) as follows: 
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tg 0.7281 0.5229| 0.4146 
[Boxee = | -0.5351 _0.7281/—0.1282 | . (109) 
Copi (opt 0.1282 —0.4146| 0.0316 


The controllability Gramian Kr?) and the observability Gramian wer o are given as follows: 


(opt) _[ 0.5100 —0.0870 
Ko =| 60670 0.4901 me) 
(opt) _[ 0.4901 —0.0870 
ae Eo 0.5100 | ` a) 


We have to note that the controllability Gramian Kor?) and the observability Gramian were) 
satisfy the sufficient condition of the absence of limit cycles given in Eq. (106) with 


Bopt = diag(0.9803, 1.0201). (112) 


Therefore, (Aopt, bopt- Copt, dopt) is the minimum L»-sensitivity realization without limit 
cycles. 


We demonstrate the absence of limit cycles in the minimum L»-sensitivity realization by 
observing its zero-input response. We calculate the zero-input responses of the minimum 
L»-sensitivity realization and the dierct form II, setting the initial state as x(0) = [0.8 — 0.8]’. 
We let the dynamic range of signals to be [—1, 1) and adopt two’s complement as the overflow 
characteristic. The zero-input responses are shown in Fig. 8(a) and 8(b). We assume that each 
filter coefficient and signal have 16[bits] fixed-point representation, of which lower 14[bits] 
are fractional bits. In this numerical example, the overflow of the state variables occurs in 
both cases. It is desirable that the effect of the overflow is decreasing since the digital filter 
H(z) in Eq. (108) is stable. For the minimum L»-sensitivity realization synthesized by our 
proposed method, the state variables x(n) and x2(n) converge to zero after the overflow, as 
shown in Fig. 8(a). Therefore, there are no limit cycles. On the other hand, for the direct form 
II, a large-amplitude autonomous oscillation is observed as shown in Fig. 8(b). Therefore, 
the direct form II generates the limit cycles. 


5.2.2. High-order digital filters 
We can demonstrate the validity of the proposed method for also high-order digital filters. 


Consider a fourth-order band-pass digital filter H(z) given by 


__ 0.0178 — 0.0252z~! + 0.0173z~* — 0.0252z~$ + 0.0178z~4 


H 
(2) 1 — 2.6977z—1! + 3.541027? — 2.3340z—3 + 0.7497z—4 


(113) 


The frequency response of the digital filter H(z) in Eq. (113) is shown in Fig. 9. We obtain 
the limit cycle free minimum L»-sensitivity realization (Aopt, bopt, Čopt, dopt) by successive 
approximation method: 
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0.6028 0.6394 0.1512 0.0655] 0.0344 

Ali —0.6360 0.7461 —0.0655 —0.0297|—0.0153 
are = | —0.0806 —0.0283 0.6028 0.6360] 0.3950} . (114) 

Copt |dopt 0.0283 0.0118 —0.6394 0.7461)—0.1423 

| 0.3950 0.1423 0.0344 0.0153] 0.0178 


The controllability Gramian Kor?) and the observability Gramian wer 9 are given as follows: 


0.3531 —0.0004 0.2470 0.0155 


KP) — —0.0004 0.3563 —0.0157 0.2470 (115) 
0 | 0.2470 —0.0157 0.5309 0.0006 
0.0155 0.2470 0.0006 0.5264 
0.5309 —0.0006 0.2470 0.0157 
x,(opt) | —0.0006 0.5264 —0.0155 0.2470 
Wor -= 0.2470 —0.0155 0.3531 0.0004 (116) 


0.0157 0.2470 0.0004 0.3563 


We have to note that the controllability Gramian Kr?) and the observability Gramian were) 
satisfy the sufficient condition of the absence of limit cycles given in Eq. (106) with 


Bopt = diag(1.2261, 1.2155, 0.8156, 0.8227). (117) 


Therefore, (Äopt Bape, Copt, dopt) is the minimum L»-sensitivity realization without limit 
cycles. 


We demonstrate the absence of limit cycles in the minimum L»-sensitivity realization 
by observing its zero-input response. We calculate the zero-input responses of the 
minimum L»-sensitivity realization and the direct form II, setting the initial state as x(0) = 
[0.9 0.9 0.9 0.9]7. We let the dynamic range of signals to be [—1,1) and adopt two’s 
complement as the overflow characteristic. The zero-input responses are shown in Fig. 
10(a) and 10(b). We assume that each filter coefficient and signal have 16[bits] fixed-point 
representation, of which lower 13[bits] are fractional bits. In this numerical example, the 
overflow of the state variables occurs in both cases. Also in this case, we can confirm that 
the minimum Lp-sensitivity realization does not generate limit cycles. On the other hand, a 
large-amplitude autonomous oscillation is observed in the zero-input response of the direct 
form I. 
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Figure 9. Frequency Response of digital filter H(z) in Eq. (113). 
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Figure 10. Zero-input responses of digital filter H(z) in Eq. (113). 


6. Conclusions 


This chapter presents analytical approach for synthesis of the minimum Lp-sensitivity 
realizations for state-space digital filters. The contributions of this chapter are summarized 
as follows. 


Section 3 presents closed form solutions to the L-sensitivity minimization problem for 
second-order state-space digital filters. We have shown that the Lo-sensitivity is expressed 
by a linear combination of exponential functions, and we can synthesize the minimum 
Lo-sensitivity realization by only solving a fourth degree polynomial equation, which can 
be solved analytically. 


Section 4 reveals that the Ly-sensitivity minimization problem can be solved analytically for 
arbitrary filter order if second-order modes are all equal. We derive a general expression of 
the transfer function of digital filters with all second-order modes equal. We show that the 
general expression is obtained by a frequency transformation on a first-order prototype FIR 
digital filter. 
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Section 5 proves the absence of limit cycles of the minimum L»-sensitivity realization from 
the view point of relationship between the controllability and observability Gramians. The 
minimum L»-sensitivity realizations were originally known to be low-coefficient sensitivity 
filter structures. We have succeeded in discovering the novel property of the minimum 
Lp-sensitivity realizations. 
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1. Introduction 


Digital filters find wide variety of applications in modern digital signal processing systems [1, 2]. As 
a result of the recent progress in such systems, there is an ever growing demand for sharp transition 
band digital filters. These narrow transition bandwidth digital filters are usually designed by using the 
frequency response masking (FRM) approach [3]. The computational efficiency of the FRM technique 
makes it suitable for different applications, e.g. in audio signal processing and data compression [4]. 


Practical design of digital filters is based on optimization for satisfying the given design specifications 
together with the hardware architecture. However, the optimization may be carried out in terms of 
fixed configurations but variable multiplier coefficient values. On the other hand, the problem may 
concern the optimization of the hardware architecture without taking the multiplier coefficient values 
into consideration. 


In order to optimize the given design specifications, the multiplier coefficient values can be determined 
in infinite precision by using hitherto optimization techniques. However, in an actual hardware 
implementation of the digital filters, the infinite precision multipliers should be quantized to their 
finite precision counterparts, but these finite precision multiplier coefficients may no longer satisfy 
the given design specifications. Consequently, from a hardware implementation point of view, there is a 
need for finite precision optimization techniques, capable of finding the optimized digital filter rapidly 
while keeping the computational complexity at a desired level. In principle, there exist two different 
techniques for the optimization of digital filters, namely, gradient-based and heuristic optimization 
approaches. 


Gradient-based optimization techniques have been studied widely. In [5], an integer programming 
technique was developed for the optimization of digital filters over a discrete multiplier coefficient 
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space. In [6], a Remez exchange algorithm was used for the optimization of FRM finite impulse 
response (FIR) digital filters and it was shown that this algorithm may provide a speed advantage over 
the linear programming approach. However, both these techniques suffer from sub-optimality problems. 
In [7], unconstrained weighted least-squares criterion was used to develop another technique for the 
optimization of digital filters. Convex optimization approaches such as semi-definite programming [8] 
and second-order cone programming [9] have also been applied to the optimization of digital filters. 
However, if a large number of constraints are present, these optimization techniques may become 
computationally inefficient in terms of time consumption and speed. 


Heuristic optimization algorithms have emerged as promising candidates for the design and discrete 
optimization of digital filters, particularly due to the fact that they are capable of automatically finding 
near-optimum solutions while keeping the computational complexity of the algorithm at moderate 
levels. Simulated annealing (SA) and genetic algorithms (GAs) were widely used in the design and 
optimization of digital filters [10-12]. Particle swarm optimization (PSO) and seeker optimization 
algorithm (SOA) are two newly developed algorithms suitable for the optimization of various digital 
filters due to their few number of implementation parameters and high speed of convergence [13, 14]. It 
was shown that SOA has advantages over PSO in terms of the speed of convergence and global search 
ability [15]. Tabu search (TS) [16], ant colony optimization (ACO) [17], immune algorithm (IA) [18] 
and differential evolution (DE) [19, 20] are alternative candidates for the optimization of digital filters. 
All the foregoing techniques allow a robust search of the solution space through a parallel search in 
all directions without any recourse to gradient information. However, the aforementioned techniques 
were developed for infinite precision optimization of digital filters which require the user to perform a 
quantization step for a hardware implementation. 


In [21-23], a technique was developed for finite-precision design and optimization of FRM digital 
filters using GAs. finite-precision optimization of FRM FIR digital filters using PSO was studied in 
(24, 25] and finite-precision optimization of infinite impulse response based (IIR-based) FRM digital 
filters was studied in [26, 27]. PSO was originally proposed by Kennedy and Eberhart in 1995 as a new 
intelligent optimization algorithm which simulates the migration and aggregation of a flock of birds 
seeking food [28]. It adopts a strategy based on particle swarm and parallel global random search, that 
may exhibit superior performance to other intelligent algorithms in computational speed and memory. 
In PSO, a potential candidate solution is represented as a particle in a multidimensional search space, 
where each dimension represents a distinct optimization variable. The particles in the multidimensional 
search space are characterized by corresponding fitness values. They make movements in the search 
space towards regions characterized by high fitness values. 


The conventional FRM digital filters incorporate FIR interpolation digital subfilters. These digital 
subfilters are usually of high orders, rendering the resulting overall FRM digital filters as not 
economical, since the resulting digital filters occupy large chip areas and consume high amounts of 
power in their VLSI hardware implementations. In general, the multiplication operation is the most 
cost-sensitive part in such an implementation. Therefore, there is every incentive to reduce the number 
of multiplication operations in the digital filter realization. This problem may be circumvented by 
employing IIR interpolation digital subfilters [29, 30]. 


There is a vast body of literature available for the design and optimization of digital IIR filters [31-33]. 
However, all the aforementioned designs are based on the exact transfer function coefficients which 
leads to an uneconomical hardware realization of such filters. In order to realize the constituent 
IIR interpolation digital subfilters on a hardware platform, the bilinear-lossless-discrete-integrator 
(bilinear-LDJ) digital filter design approach is employed [34]. These digital subfilters are realized as a 
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sum/difference of a pair of bilinear-LDI digital allpass networks. The salient features of the bilinear-LDI 
digital filters are that they lend themselves to fast two-cycle parallel digital signal processing speeds, 
while being minimal in the number of digital multiplication operations (and, practically, minimal in 
number of digital addition and unit-delay operations). 


The starting point in the design of FRM digital filters is to find the multiplier coefficients constituent in 
the FRM digital filter in infinite precision by using the hitherto gradient-based optimization techniques 
(e.g. Parks-McClellan approach [35] for FIR digital filters) followed by a quantization step. The 
quantization can be performed by constraining the multiplier coefficients values to conform to certain 
number systems such as the signed power-of-two (SPT) system. SPT is a computationally efficient 
number system which can further reduce the hardware complexity of the FRM IIR digital filters. In 
this number system, each multiplier coefficient is represented with only a few non-zero bits within its 
wordlength, permitting the decomposition of the multiplication operation into a finite series of shift and 
add operations. Digital filters incorporating SPT multiplier coefficient representation are commonly 
referred to as multiplierless digital filters [36]. However, the SPT representation of a given number is 
not unique, resulting in redundancy in the multiplier coefficient representation. This redundancy can 
adversely affect the corresponding computational complexity due to recourse to compare operations 
repetitively. 


The canonical signed digit (CSD) number system is a special case of the SPT number system 
which circumvents the above redundancy problem by limiting the number of non-zero bits in the 
representation of the multiplier coefficients. It is usually used in combination with subexpression 
sharing and elimination, which in turn results in substantial reduction in the cost of the VLSI hardware 
implementation of the digital filters [37]. In CSD number system, no two (or more) non-zero bits can 
appear consecutively in the representation of the multiplier coefficients, reducing the maximum number 
of non-zero bits by a factor of two in terms of shift and add operations [38]. 


After multiplier coefficient quantization, the resulting FRM digital filter may no longer satisfy the 
given target design specifications. Therefore, the next step in the design of FRM digital filters is to 
perform a further optimization to make the finite precision FRM digital filter to conform to the design 
specifications. This can be achieved by resorting to a finite-precision optimization technique such as 
PSO. 


A direct application of the conventional PSO algorithm to the optimization of the above FRM digital 
filters gives rise to three separate problems: 


e The first problem arises because in the course of optimization, the multiplier coefficient update 
operations lead to values that may no longer conform to the desired CSD wordlength, etc. (due 
to random nature of velocity and position of particles). This problem is resolved by generating 
indexed look-up tables (LUTs) of permissible CSD multiplier coefficient values, and by employing 
the indices of LUTs to represent FRM digital filter multiplier coefficient values. 


e The second problem stems from the fact that in case of FRM IIR digital filters, the resulting FRM 
IIR digital filters may no longer be bounded-input-bounded-output (BIBO) stable. This problem can 
be resolved by generation and successive augmentation of template LUTs until the BIBO stability 
constraints remain satisfied [23]. 


e Finally, the third problem arises because even in case of having indexed LUTs, the particles may go 
over the boundaries of LUTs in course of optimization (due to the inherent limited search space). 
This can be resolved by introducing barren layers. A barren layer is a region, with a certain width 
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and certain entries, which is added to the problem space such that the particles tend to shy away 
from such a region. The width of the barren layers is calculated based on a worst case scenario that 
may happen in the particles movements in the search space. However, the entries of barren layers 
are different for different problems and depend on the topology of the search space and the fitness 
function used in the problem. 


This chapter discusses in detail the design, realization and discrete PSO of FRM IIR digital filters. FRM 
IIR digital filters are designed by FIR masking digital subfilters together with IIR interpolation digital 
subfilters. The FIR filter design is straightforward and can be performed by using hitherto techniques. 
The IIR digital subfilter design topology consists of a parallel combination of a pair of allpass networks 
such that its magnitude-frequency response matches that of an odd order elliptic minimum Q-factor 
(EMQF) transfer function. This design is realized using the bilinear-LDI approach, with multiplier 
coefficient values represented as finite-precision CSD numbers. 


The above FRM digital filters are optimized over the discrete multiplier coefficient space, resulting in 
FRM digital filters which are capable of direct implementation in digital hardware platform without any 
need for further optimization. A new PSO algorithm is developed to tackle three different problems. In 
this PSO algorithm, a set of indexed LUTs of permissible CSD multiplier coefficient values is generated 
to ensure that in the course of optimization, the multiplier coefficient update operations constituent in the 
underlying PSO algorithm lead to values that are guaranteed to conform to the desired CSD wordlength, 
etc. In addition, a general set of constraints is derived in terms of multiplier coefficients to guarantee 
that the IIR bilinear-LDI interpolation digital subfilters automatically remain BIBO stable throughout 
the course of PSO algorithm. Moreover, by introducing barren layers, the particles are ensured to 
automatically remain inside the boundaries of LUTs in course of optimization. 


2. The conventional PSO algorithm 


Let us consider an optimization problem consisting of N design variables, and let us refer to each 
solution as a particle. Let us further consider a swarm of K particles in the N-dimensional search space. 
The position of the k-th particle in the search space can be assigned a N-dimensional position vector 
Xe = {Xk Xk- -+ Xk. In this way, the element xz; (for j = 1,2,...,N) represents the j-th 
coordinate of the particle Xx. 


The PSO optimization fitness function maps each particle X, in the search space to a fitness 
value. In addition, the particle X, is assigned a N-dimensional velocity vector Vi = 
{Uk Vk2; -< +, URN}. The PSO optimization search is directed towards promising regions by 
taking into account the velocity vector Vą together with the best previous position of the k-th 
particle Xpest, = {XbestysXbestysr - -7 Xbestıy f> and the best global position of the swarm Gpest = 
{Shesty, Shesta «+» Zbesty f (i.e. the location of the particle with the best fitness value). 


The conventional PSO is initialized by spreading the particles X% through the search space in a random 
fashion. Then, the particles make movements through the search space towards regions characterized 
by high fitness values with corresponding velocities V,. The movement of each particle is governed 
by the best previous location of the same particle Xpe5¢,, and by the global best location Gpest. The 
velocity of particle movement is determined from the previous best location of the particle, the global 
best location, and the previous velocity. 
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Figure 1. Movement of Particles in PSO Algorithm 


The velocity and position of each particle in the i-th iteration throughout the course of PSO are updated 
in accordance with the equations: 


TEE E ee fa) it 
Oj = WO; + C171 (Chest, Xkj ) + C272 (Siest, Xj ) () 


: i ` b 
if Ukj < Umin å} Ukj = Vmin 
if Oj > Umax i Ukj = Vmax 


i _ ,i-1 i 
Xkj = Xij + OK; (2) 


The parameter w represents an inertia weight; c4 and cp are the correction (learning) factors, and r4 and 
r2 are random numbers in the interval [0,1]. The velocity is limited between Vmin and Vmax to avoid 
very large particle movements in the search space, where Vmin < 0 and Umax > 0. Fig. 1 illustrates 
how the particles move in a two-dimensional search space (N = 2). In this figure, two particles are 
present in the swarm, i.e. K = 2. 


The first term in the right hand side of movement update Eqn. (1), weighted by w, signifies the 
dependence of the current particle velocity on its value in the previous iteration. The second term, 
weighted by c1, signifies an attractor to pull the particle towards its previous best position. The third 
term, weighted by c2 controls the movement of the particle towards the global best position. 


In addition to the update Eqns. (1) and (2), one can limit the coordinates in a particle between two user 
defined values x jmin Nd Xj,,,, 1N Order to limit the search space. However, This operation increases the 


mi max 


complexity and consumes time. 
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Figure 2. FRM Digital Filter Block Diagram 
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Figure 3. Block Diagram Representation of Frequency-Response Masking 
3. The conventional FRM design approach 


3.1. Design of lowpass FRM digital filters 


The block diagram in Fig. 2 shows a conventional FRM digital filter, where H,(z) represents a FIR 
interpolation lowpass digital subfilter, and where Hp (z) represents a power complementary counterpart 
of Ha (z) in accordance with 


|Ha(el!)? + |H (el)? =1 (3) 


Here, z represents the discrete-time complex frequency, and w represents the corresponding 
(normalized) real frequency variable. Moreover, Fo(z) and F(z) represent FIR masking digital 
subfilters, while Ha(z™) and H,(z™) represent M-fold interpolated versions of H,(z) and Hy(z), 
respectively. In case of FIR digital interpolation subfilters, for a linear-phase filter H,(z) of order 
Neyp, the relationship between Hp (z) and Ha(z) is as follows: 


Hy(z) = z(Neiet1)/2 _ H (z) (4) 


and hence H(z) can be implemented by subtracting the output of Ha (z) from the delayed version of 
the input, as shown in Fig. 3. 


The FRM digital filter in Fig. 2 has an overall transfer function 


H(z) = Ha(z™)Fo(z) + Hy (z™)F, (z) (5) 
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Filter | Passband Edge Stopband Edge 


H(z) an Purto 
Case I Fo(z) aunt Wa 2(I; ronz Wa 
a -n 
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Case I Fo(z) 2(i.= Un + Wa a Wa 
oe ee 


Table 1. Edge Frequencies of the Overall FRM FIR filter and Masking Subfilters 


The masking digital subfilters Fọ(z) and F,(z) are employed to suppress the unwanted image bands 
produced by the interpolated digital subfilters Ha(z™) and H,(z™). The masking filters are made to 
have equal order (by zero padding) in order to ensure that their phase characteristics are similar. The 
corresponding interpolated digital subfilters H,(zM) and H,(z™) can realize transition bands which 
are a factor of M sharper than those of Hg(z) and H(z), without increasing the number of required 
non-zero digital multipliers. The magnitude frequency-response of the various subfilters incorporated 
by the FRM digital filter design approach are shown in Fig. 4. 


Here, Case I design is when the transition band of H(z) is extracted from that of H,(z™) and Case II 
design is when the transition band of H(z) is extracted from that of H,(z™). The edge frequencies of 
the overall digital FRM filter and its constituent subfilters are given in Table 1, where Iz, represents the 
number of image lobes to be masked given by: 


| Mwy | 
— Case I 
27 


IL = (6) 
| Mw | Case II 


27 


where | | denotes the largest integer from the lower side, and | | signifies the smallest integer from the 
upper side. 


3.2. Design of bandpass FRM digital filters 


In general, it is possible to extend the conventional FRM approach for the design of bandpass or 
bandstop FRM digital filters. However, the resulting FRM digital filters are constrained to have identical 
lower and upper transition bandwidths. In [39], this restriction was relaxed by realizing the bandstop 
FRM FIR digital filter as a parallel combination of a corresponding pair of lowpass and highpass FIR 
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Figure 4. Magnitude Frequency-Response of FRM Digital Filter. (a) Magnitude Frequency-Response of the Bandedge-Shaping 
Digital Subfilters H,(z) and H,(z). (b) Magnitude Frequency-Response of the M-Interpolated Complementary Digital Subfilters 
H,(z™) and H,(z™). (c) Magnitude Frequency-Response of the Masking Digital Subfilters Fy(z) and F,(z) for Case |. (d) 
Magnitude Frequency-Response of the Overall FRM Digital Filter H(z) for Case |. (e) Magnitude Frequency-Response of the 
Masking Digital Subfilters Fy(z) and F (z) for Case Il. (f) Magnitude Frequency-Response of the Overall FRM Digital Filter H(z) 


for Case II [3]. 
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Figure 5. Bandpass FRM Digital Filter Block Diagram 


digital filters. The latter lowpass and highpass FRM digital filters were obtained using a variation of the 
conventional FRM approach. 


Let the desired bandpass FRM digital filter H(z) have a lower transition bandwidth which is not 
identical to its upper transition bandwidth. H(z) can be realized as a cascade combination of a pair 
of lowpass and highpass FRM digital filters, so that 


H(z) = Hıp(z)Ħnp(z) (7) 


where H),(z) represents a lowpass and Hpp(z) represents a highpass FRM digital filter. In this way, 
Hip (z) and Hpp (z) can be obtained with the help of Eqn. (5) as 


(2) Fo, (z) + Hp, (2) Fi, (2) (8) 
Hyp (z) = Hayy (2™)Fo,, (2) + Ho, (2) Fang (2) (9) 


The lower transition bandwidth is governed by the constituent transition bandwidth of the highpass 
FRM digital filter, while the upper transition bandwidth is governed by the constituent transition 
bandwidth of the lowpass FRM digital filter. The realization for bandpass FRM digital filter are as 
shown in Fig. 5. 


4. Design of FRM digital filters incorporating IIR interpolation digital 
subfilters 


In the case of FRM IIR digital filters, H(z) and H,(z) (in section 3) act as IIR interpolation digital 
subfilters. The masking filters Fg(z) and F4 (z) are not changed (i.e. they are still equal order FIR digital 
filters). Therefore, Eqn. (5) is still valid for the FRM IIR digital filter. 


The IIR interpolation digital subfilter Ha (z) is chosen to have an odd order Ny ;r. Odd-ordered elliptic 
transfer functions can be represented as a sum of or difference between two allpass transfer functions 
[40]. Therefore, Ha (z) can be realized as the addition of two allpass digital networks Gg(z) and G4 (z) 
as follows: 


Go(z) + Gi(z) 
2 
where Go(z) is odd-ordered and Gj(z) is even-ordered. The interesting fact is that the difference 


between Go(z) and G4 (z) results in a filter that is power complementary to H,(z), and can subsequently 
be used as the power complementary interpolation digital subfilter H, (z) as in the following: 


H,(z) = (10) 


Go(z) — G1 (z) 


H(z) = 7 


a1) 
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Figure 6. Block Diagram of Interpolation and Complementary Filters as a Parallel Combination of Two Allpass Networks 
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Figure 7. FRM Digital Filter Realization in Terms of Allpass Digital Networks Go(z) and G1 (z) 


It can be easily verified that H,(z) and H(z) are power complementary digital filters [29], i.e. they 
satisfy Eqn. (3). In addition, it is well known that this structure halves the number of multiplier 
coefficients required for the implementation of FRM digital filters and therefore is the most economical 
realization since it requires a total of only Nj; multiplier coefficients to realize both H,(z) and H,(z). 
The overall transfer function of H(z) given by Eqn. (5) can be expressed as: 


H(z) — Go(z™) a Gy GTE | Go(z™) z Gy(z™) F(z) (12) 


The block diagram in Fig. 6 shows the IIR interpolation digital subfilters H,(z) and H;,(z) realized 
as a parallel combination of two allpass networks. It should be noted that if H(z) is a lowpass filter, 
H,(z), which is the power complementary of H,(z), is a highpass filter. Fig. 7 shows an overall FRM 
IIR digital filter realization. 


One may rearrange the structure in Fig. 7 by using Eqns. (10-11). This can be performed by defining 
two digital subfilters as follows: 


(13) 


(14) 


Then H(z) in Eqn. (12) simplifies to: 


H(z) = Go(z™) A(z) + Gi(z™)B(z) (15) 
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Figure 8. Alternative Structure of the Overall FRM IIR Digital Filter 


Fig. 8 shows the block diagram representing Eqn. (15). 


The advantage of realizing the FRM IIR digital filter as shown in Fig. 8 is that two adders shown in Fig. 7 
are removed and they are no longer required. This subsequently simplifies the hardware implementation 
of the overall FRM IIR digital filter. However, it should be noted that the FIR masking digital subfilters 
Fo(z) and F; (z) are made to be equal order using zero padding, and this results in the masking filters 
being moderately sparse. This is not the case when A(z) and B(z) are used instead. Therefore, the gain 
in hardware that could be achieved by using the realization in Fig. 8 is offset by a greater number of 
non-zero multiplier coefficients required in the realization of FRM IIR digital filters. 


5. Realization of IIR interpolation digital subfilters using Elliptic Filters with 
Minimum Q-factor (EMQF) 


Bilinear-LDI transformation falls into the category of digital filter realization techniques that transform 
an analog reference filter to its digital counterpart. Therefore, in order to determine the multiplier 
coefficient values of the IIR interpolation digital subfilters Ha (z) and Hp (z) constituent in the FRM IIR 
digital filter, a suitable analog reference filter Ha (s) and its power complementary analog filter H, (s) 
have to be determined, where s is the analog frequency domain variable. Once H,(s) and Hp(s) have 
been determined, the interpolation digital subfilters H4 (z) and Hj(z) are derived by using bilinear-LDI 
technique (see Section 6). 


EMOQF filters have several advantages for the design of FRM IIR digital filters. The squared ripple 
in the passband region of H(z) and the squared ripple in the stopband region of Hp(z) are equal as 
indicated by Eqn. (3). On the other hand, the squared ripple in the stopband region of H,(z) and the 
squared ripple in the passband region of H;,(z) are equal. In addition, depending on whether the design 
specifications require a Case I or Case II FRM technique, either H,(z) or Hy(z) could determine the 
maximum passband and stopband ripple of the overall FRM IIR digital filter H(z). Consequently, 
the interpolation filter Ha (z) is chosen to have equal passband and stopband squared tolerances. In 
this way, the resulting H)(z) also displays equal passband and stopband squared tolerances. These 
characteristics can be generalized for the analog reference subfilters Ha (s) and H, (s). Therefore, there 
is a need for an analog reference filter H,(s) that together with its power complement H, (s) can exactly 
satisfy the passband and stopband relations in the FRM UR filter. EMQF filters can successfully comply 
with the specifications present in the FRM IIR filter design. In addition, an EMQF transfer function 
can be easily designed by using bilinear-LDI transformation technique or any other structure consisting 
of two digital allpass networks in parallel. Furthermore, filters having EMQF transfer functions are 
minimally sensitive to component variations. 
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Despite all the advantages of EMQF filters, they suffer from not being able to independently specify 
passband and stopband ripples [41],[42] of the filter. Additionally, EMQF filters have exceedingly low 
passband attenuation. 


All the poles of an EMQF transfer function reside on a circle in the s domain rendering them to have 
equal magnitudes. Given a squared passband and stopband tolerance of 6) and dq, respectively, for an 
EMOQF filter, the passband ripple Ap and minimum stopband attenuation A, can be obtained as follows 
[43]: 


Ap = —10log(1 — dy) (16) 
Aq = —10log(d,) (17) 


The required passband and stopband edge frequencies for the analog reference filter Ha(s) can be 
determined using design specifications along with Table 1. Frequency wrapping from digital to analog 
domain, and vice versa, has to be taken into account in accordance with: 


i= Z tan( 247) (18) 


where Q 4 is the analog frequency variable, where wg is the digital frequency variable, and where T is 
the sampling period. 


Once the transfer function of the analog reference filter H,(s) is determined, it is represented as a sum 
of two allpass analog filters Gg(s) and G4 (s). In addition, H,(s), which is the power complementary 
of H,(s) is represented as the difference of Go(s) and G(s). The poles of Go(s) and G(s) are 
determined by cyclically distributing the poles of the reference filter H,(s) [43]. In the next section, 
belinear-LDI design technique is used to transform the two allpass networks Go(s) and G(s) into 
digital domain. 


6. Implementation of EMQF interpolation subfilters using bilinear-LDI design 
approach 


In this section, the design procedure in [34, 44] is briefly explained to design and implement digital 
filters Go (z) and G4 (z) using the the bilinear-LDI approach. This approach transforms analog reference 
filters Gg(s) and G4 (s) to obtain their digital filter counterparts Gg(z) and G4 (z). 


The bilinear frequency transformation maps the analog frequency variable s to its digital domain 
counterpart z in accordance with: 


a 2z-1 
~ Tz+1 


where T represents the sampling period, for mapping the transfer function of a prototype reference 
filter from the analog domain to the digital domain. The bilinear transform maps the left half of the 
complex s-plane to the interior of the unit circle in the z-plane. Therefore, BIBO stable filters in the 
s domain are converted to filters in the z domain which preserve that stability. Similarly, if the analog 
reference filter is minimum-phase, the previous characteristic of bilinear transform guarantees that the 
resulting digital filter is also minimum-phase. It also preserves the sensitivity properties of the analog 


(19) 
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reference filter. However, bilinear transform may result in a digital filter that has delay-free loops in 
its implementation. Unfortunately, delay-free loops prevent the implementation of a digital filter to be 
realizable in hardware platform. 


The LDI frequency transformation ensures the absence of delay-free loops in the digital implementation 
and is given by 


g=} (22-274) (20) 


The LDI frequency transformation maps the hardware implementation of the analog reference filter to 
digital domain. While the LDI frequency transformation guarantees that there are no delay-free loops in 
the implementation of the digital filter, it does this to the cost of resulting in a digital filter having poor 
magnitude-frequency responses. Moreover, it is incapable of preserving the BIBO stability properties 
of the analog reference filter. 


The bilinear-LDI approach is a combination of the two above mentioned realization techniques. In 
bilinear-LDI transform, a precompensation is performed to the reference analog filter. Then, the 
conventional LDI design technique is applied to a network resulting from the precompensated analog 
prototype filter. The precompensation is such that the application of the LDI design technique results 
in a filter that exactly matches the bilinear frequency transform of the uncompensated analog prototype 
filter. 


The resulting bilinear-LDI digital filters have several desirable features from a hardware realization 
point of view. They are minimal in the number of digital multiplication operations. Although they are 
not minimal in the number of digital adders and unit-delays, the additional adders and the additional unit 
delay lead to certain advantages when the concept of generalized delay unit is used for the realization 
of the network [34]. Moreover, The bilinear-LDI digital filters lend themselves to fast two-cycle 
parallel digital signal processing speeds and they exhibit exceptionally low passband sensitivity to their 
multiplier coefficient values, resulting in small coefficient wordlengths. 


As discussed in Section 5, the analog reference filter H,(s) is decomposed into two allpass analog 
networks Go(s) and G;(s). The digital allpass networks Gg(z) and G4 (z) are obtained from Go(s) and 
G(s) using the bilinear-LDI design approach. 


It should be pointed out that Go(s) is an odd-ordered allpass function. Therefore, it has a pole on the 
real axis in the s domain. On the other hand, G4 (s) ends up having an even-ordered allpass function. It 
is well known that an allpass transfer function can be written in the general form [34]: 


G(s) = (21) 


where P(s) is a Hurwitz polynomial of order, say, ñ . Moreover, P(s) can be expressed as: 
P(s) = EvP (s) + OdP(s) (22) 


where EvP(s) denotes the even and OdP(s) denotes the odd part of P(s). 
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K 
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Va, (s) Va, (s) 
Figure 9. Signal Flow Graph of G(s) 
By simple manipulation of Eqns. (21) and (22) one can get 
~1—Z(s) 
G(s) = K——— 23 
(s) 1+ Z(s) 23) 
Here, K = 1 or —1, and Z(s) is a realizable reactive impedance given by 
OdP 
EP = for even ñ 
Z(s) = (24) 
EVEN?) for odd ñ 
OdP(s) 


where ñ is the order of G(s) (odd when realizing Go (s) and even when realizing G4 (s)). The impedance 
Z(s) has a zero at s = 0 for even ñ and a pole at s = 0 for odd ñ, while having a zero at s = œœ both 
for even ñ and for odd ñ. 


The bilinear-LDI digital realization of G(s) is achieved by using the following steps: 


e The transfer function G(s) is decomposed in the form 


G(s) = K[1 — 2g(s)] (25) 
where 
g(s) = is (26) 


Here, G(s) can be realized as the transfer function of the signal-flow graph in Fig. 9. 


Furthermore, g(s) represents a lowpass or highpass analog filter that can be realized as the transfer 
function of the voltage divider network in Fig. 10. 
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Figure 10. Voltage Divider Circuit for g(s) 


Figure 11. Realization of Impedance Z(s) 


Finally, Z(s) represents realizable reactances (consisting of capacitors and inductors only) and can 
be decomposed into its Foster II canonical form, as in Fig. 11, in accordance with 


(27) 


m sC 


Y(s) = Es 
(s) SC T +h 26 


pd 2 28 
Litl 28) 


where m = ñ/2 for even ñ and m = (ñ + 1)/2 for odd ñ, and where C; represent capacitances 
and L; represent inductances (for i = 1,2, ..., m), and inductor L4 is only present for even ñ. 


e The impedance Z (s) in Fig. 11 is substituted into Fig. 10 and the precompensation is applied to the 
resulting network. This amounts to a modification of circuit elements in accordance with: 


(29) 
The resistance in rọ in Fig. 10 is modified to: 


ro =z} ro (30) 
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and 


in 
f G+ 


with r9 = 10 and for i = 2,3, ..., m. 


(31) 


(32) 


(33) 


(34) 


e Since the voltage/current signal-flow graph of the precompensated network [34] consists of analog 
integrators only and it has no analog differentiators, it can be used for bilinear-LDI realization 
method. Therefore, the analog integrators in the signal-flow graph of the precompensated network 
are replaced by LDI digital integrators, and by impedance-scaling, the resulting network is scaled 


by z~? to eliminate any half-delay elements. The resulting digital network is displayed in Fig. 12. 


The multiplier coefficients in Fig. 12 are as follows: 


N 

= 
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A 
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Figure 12. Realization of the Bilinear-LDI Digital Allpass Network G (z) [34 


(35) 


(36) 
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7. Constraints for guaranteed BIBO stability 


In order for the FRM digital filter consisting of CSD multiplier coefficients ifp m to be BIBO stable, 
it is both necessary and sufficient for the bilinear-LDI IIR interpolation digital subfilters H,(z) and 
H,,(z) to be BIBO stable. Likewise, in order for the interpolation digital subfilters H,(z) and H;(z) to 
be BIBO stable, it is both necessary and sufficient for the bilinear-LDI allpass digital networks Go (z) 
and G4 (z) to be BIBO stable. In this way, it is required that the bilinear-LDI digital allpass networks 
Go(z) and G4 (z) remain BIBO stable throughout the course of the PSO algorithm. 


In the course of PSO algorithm, the infinite-precision multiplier coefficients my, and mc, can only take 
quantized values ız, and fic, that belong to CSD(L,1, f). In order for the bilinear-LDI digital allpass 
networks Go(z) and G;(z) to remain BIBO stable, it is required that the values of the corresponding 
quantized reactive elements £; and C; remain positive [45] in the course of optimization. This is due to 
the properties of the bilinear frequency transformation from analog to digital domain. In order to find 
the conditions for BIBO stability and in accordance with Eqns. (35) and (36), one has: 


al F 
ĉi = — (37) 
11; 
Al F 
C= (38) 
MGC; 


Moreover, in accordance with Eqns. (31-34), one has: 


EA (39) 
A T2 
ay A T T m GI 
1 Cy T 2 t A t Z m (40) 
4L 0 CG; + i 
a 272 
L=t C+ aE (41) 
Lat Ê: 
at ce 
1 
= — (42) 
"Gt =- 


where 1 = œ for odd-ordered allpass network Go(z). 


By substituting Eqns. (37) and (38) into Eqns. (39-42), and by solving the resulting equations for the 
reactive elements L; and C;, one can obtain 


(43) 
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4 y 1 
T4 -— -ñ —4 —2 
7 Mc, 1 3 ae — Tic, 
Cc, = —.—— sv (44) 


Li = 7 (45) 
ĉ = o __. (46) 


From Eqns. (43-46), L; > 0 and C; > 0 provide that 


mz, >0 (47) 
mz, > 0 (48) 

4 
fic; < = (49) 

ML; 

n 4 
tic, < (50) 

m 1 
my, T 4 L A + 2 


i=2 Thi, Me; 


Then, in order to make the CSD FRM digital filter BIBO stable, it is necessary and sufficient to 
choose the values of the multiplier coefficients tippy €E CSD(L,I,f) such that the inequality 
constraints (47-50) are satisfied. The equations and corresponding condition required for BIBO stability 
are summarized in Table 2. 


In order to make the CSD lowpass digital IR FRM filter BIBO stable, it is necessary and sufficient 
to choose the values of the multiplier coefficients 11;,, fic, € CSD(L,1, f) such that the inequality 
constraints of Table 2 are satisfied. 


It should be pointed out that constraint (49) is most stringent when 7z, is at its largest possible value. 
Similarly, constraint (50) is most stringent when i}, , fı, and fic, are all at their largest possible values 


(while fiz, and fic, still adhere to constraint fic, < 4 (r1,,)~'). 


8. Proposed PSO of FRM IIR digital filters 


The proposed particle swarm optimization of BIBO stable FRM IIR digital filters is carried out over 
the CSD multiplier coefficient space CSD(Lo or 1,/0 or1, foor1)» Where Lo or 1 represents the multiplier 
coefficient wordlength, where [9,1 represents the maximum number of non-zero digits, and where 
foor1 represents the number of fractional part digits (for FIR or IIR digital subfilters, respectively). 
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Element Equation Inequality Constraints 
x T 
Îî n it 0 
1 fir mi, > 
ce o o oA 
a 1 4 m 1 m 1 
Cy Ir ma "L —4 A aT —2 Mc, <4 ML, +4 “fa. +2 
i=2 m, Ci i=2 m, "C; 
A T(r, fic, = 4)? 
f: pai ile He ee ae nip, > 0 
i 16m, Li 
A —4T = 
Ci - fic, < 4 (111, ) i 


fic (fiL fic, = 4) 


Table 2. Relations for Elements of Back-Transformed Reactance 


The starting point of any stochastic algorithm plays an important role in the convergence behavior of the 
optimization algorithm [46]. Therefore, it is important to generate the initial swarm in proper positions 
in the search space rather than complete random generation of the initial population. In order to achieve 
this, the following technique is employed: 


8.1. Initiation of PSO 


To start the PSO algorithm from a good position in the search space the infinite precision multiplier 
coefficient values of the seed particle are generated by using classical techniques as discussed in 
previous sections. These infinite precision multiplier coefficient values are turned into their finite 
precision counterparts by simply rounding them to their closest CSD values. This seed particle is used 
as the center of the swarm and a cloud of particles are generated randomly around the seed particle. It 
should be noted that the distance of the randomly generated particles should not be far from the seed 
particle. In this way, the initial swarm contains particles which have high chances of being near the 
optimal solution. The multiplier coefficient values of the swarm are taken from a set of CSD LUTs 
which are constructed as follows: 


8.2. FRM IIR digital filter template LUTs 


It is necessary and sufficient to choose the values of the multiplier coefficients, such that the inequality 
constraints (47-50) are satisfied. In order to achieve this, the LUTs are constructed as follows: 


e One LUT is constructed for all multiplier coefficient values ffir E€ CSD(Lo,lo, fo) for the 
masking digital subfilters Fo(z) and F; (z). The values of Lọ, lọ and fọ are determined empirically 
based on the amplitude frequency-response of the masking digital subfilters F(z) and F; (z). 


e A LUT is constructed for all multiplier coefficient values fiyr E€ CSD(L4, h, fı) for the digital 
allpass networks Go(z) and G;(z). Once again, the values of L4, 1; and fy are determined 
empirically. Also, it is expedient to assume that firr have only positive values. 


e The above CSD LUT is used to form one size-reduced LUT per the multiplier coefficient for 
digital allpass networks Go(z) and G1 (z), where each size-reduced LUT initially includes CSD 
values bounded from below by the smallest representable value belonging to CSD (L4, l4, fy), and 
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from above by the corresponding value of the finite-wordlength coefficients for the seed particle. 
The size-reduced LUTs are augmented before PSO process commences. The purpose of this 
augmentation is to ensure that the exploration space include as many of those CSD multiplier 
coefficients 7i1;,,, Mc, Mp, and fic, which still satisfy the BIBO stability constraints (47-50). 


The above constructed LUTs are used as template LUTs. There are two problems concerning the PSO 
of FRM IIR digital filters over the CSD multiplier coefficient space. To overcome these problems, the 
template LUTs must be further processed. These two problems and the way to solve them are discussed 
in the following. 


8.3. PSO indirect search method 


In PSO, the required new particle position is obtained from the previous position of the particle through 
the addition of a random (normalized) velocity value. However, by directly applying the conventional 
PSO to the above optimization over the CSD multiplier coefficients, one may obtain new particle 
positions whose coordinate values are no longer in CSD(Loor1,loor1, fOor1)- In order to overcome 
this problem, the optimization search is carried out indirectly via the indices to the LUT CSD values 
(as opposed to LUT CSD values themselves). In this way, the CSD coordinate values for each particle 
position are obtained by integer indices to the CSD LUTs. The key point in the indirect search rests 
with ensuring that the index set is closed, i.e. by ensuring that each index points to a valid CSD value 
in the LUT, and that the resulting particle in the course of PSO adheres to the prespecified CSD number 
format. 


If the velocity values are replaced by their closest integer values, the update equations become modified 
to 
Oj = [wð + erry Gn, =o cora likest, =f) (51) 
if Oj Umin i Oj = Omin 
if Oj > Omax }; Oj = Omax 
Ry = Apt + Oj (52) 


Here, £4, Ox; Lpesty» Shest) Omin and Ômax are all integer values where Ômin < O and Ômax > 0. In 
addition, w is limited in the interval [0, 0.5) (as discussed shortly). 


8.4. Barren layers 


Due to their finite length, the template LUTs inevitably lead to a bounded optimization search space. In 
order to ensure that the particles do not cross over to the outside of the search space in the course of PSO, 
the search space is constructed as a combination of two regions, namely the interior and barren layers. 
The barren layer is constructed to yield relatively low fitness values, and is represented as header and 
footer in the template LUT. There are two problems concerning the construction of the barren layers: 


1 [R] denotes rounding R to its closest integer, where R is assumed to be a real value. 
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8.4.1. barren layer entries 


The first problem in the construction of barren layers concerns how to make the fitness values in the 
barren layer relatively low. This problem can be resolved by filling the header part by unrealistically 
large, and the footer part by unrealistically small CSD multiplier coefficient values. 


8.4.2. barren layer width 


The second problem, on the other hand, concerns how to determine the width of the barren layer such 
that the particles do not cross over to the outside of the search space even under the worst case scenario. 
These two problems relate to the number of entries and the CSD values of the entries in header and 
footer parts of the template LUTs. To overcome this problem, let us consider the j-th variable in the 
k-th particle is in the boundaries of one of the template LUTs in iteration i — 1. The worst case scenario 
occurs when Xj moves toward the barren layer with the peak permissible velocities (Umax for the 


header, and v,,,;,, for the footer). If in the i-th iteration x i is in the footer: 


Phesty > Lj (53) 

Shest, > Py (54) 
and if it is in the header: 

Piesty < ty (55) 

Shest, < fij (56) 


Eqns. (53-56) show that the velocity of the particle in iteration i + 1 tends to move the particle in a 
direction opposite to the direction of the barren regions. Here, the worst case happens when r1 = r2 = 
0. In this way, the number of entries Lẹ in the footer part, and the number of entries Ly in the header 
part is determined in accordance with 


Lr = |Omin| Tr [W|Omin ] a [w[w| ôminl]] ree 
A |Onin| |Omin| 
< ; } 
= |Omin| 2 4 
= 2|Omin| (57) 
Ly = Omax + [Wmax] + [wlwomax]] +... 
a Omax Omax 
< + — gee 
S Umax 7 F 4 + 
= 20 max (58) 


Let us recall that since 0 < w < 0.5, 


if v: positive integer = [wv] < (59) 


NIS 
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In addition, after some iterations an = 0. Otherwise, if w > 0.5, on can never become zero, and 


the width of the barren layer will be infinity. 


The augmented LUTs remains fixed in the course of PSO, restricting automatic particle movement 
inside the limited search space. Modifying the index values inside each particle by adding the current 
indices to the length of the footer barren region, Lp, PSO algorithm is ready to start the optimization of 
FRM digital filters. 


9. Design methodology 


The design methodology for the proposed PSO of BIBO stable bilinear-LDI based FRM IIR digital 
filters over the CSD multiplier coefficient space can be summarized as follows: 


1. Designing the interpolation digital subfilter: the first step in determining the interpolation subfilter 
specifications is to fix the interpolation factor M from a pre-specified range. This is done in a way 
that the order of the FIR masking filters is kept minimal. Using the passband edge frequency wp 
and stopband edge frequency wa and the expressions for boundary frequencies given in Table 1, 
one can determine the filter case and calculate the approximate passband edge 6 and stopband edge 
¢ of the digital interpolation lowpass subfilter H(e/“’), for every value of the user specified range 
of interpolation factors M. The order of the FIR masking filters depends on the minimum distance 
between consecutive image replicas of either the interpolated subfilter Hy (e! Mw) or its complement 
Hy (ei Mejs Then, displacement Àm and distance Dj for each interpolation factor M are given as: 


Am = max(|(5 -LIF - 8) 60) 
z ma 2A 
Dm=M M S 


To minimize the length of FIR-masking filters, the value of M that results in the largest value of Õ m 
is chosen. This determines the optimal interpolation factor M as well as the approximate passband 
edge 6 and stopband edge ¢ of the digital interpolation subfilter H(e/”). EMQF filters have the 
property of equal square magnitude ripple size in the passband and stopband. Therefore, of the 
two ripple specifications, whichever gives the smallest tolerance in the squared magnitude response 
determines both the passband ripple Rp and stopband attenuation R4 of the interpolation digital 
subfilter H, (el ). The interpolation digital subfilter order Nj ;p is then determined using Rp, Ra, 
6 and $. Nyrr must be rounded to the nearest larger odd integer so that it can be implemented by 
a parallel combination of two allpass networks. With the order N77, and passband and stopband 
ripples Ry and Rg fixed, the ratio of the analog passband edge @ 4 and stopband edge #4 is a constant 
k given by [47] 


_ 100R: _ 4 
~~ 100-187 —1 


—log(16D) 


q=10 NIIR 
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q = qo + 22 + 1594 + 1504} 


_ [1-240]? 
P |14+ 290 


k= \/1-k5 


In order to satisfy the passband edge specification, the digital passband edge wp = 6 for Case I 
filters. The digital stopband edge wz is then determined using the analog ratio k. (Here, frequency 
warping from digital to analog domain, and vice versa, given by Eqn. (18) needs to be taken into 
account.) Similarly, wa = ¢ for Case II filters, and Wp can be determined by using ratio k. Also, 
using given ripple specifications along with the boundary frequencies described in Table 1, one can 
determine the transfer function of the FIR masking filters Fy(e/”) and F,(e/”). 


. Generation of seed FRM digital filter particle: The seed FRM digital filter particle is formed as 
follows: 


e A particle with B4 coordinates is formed in which each coordinate serves as an index of the 
corresponding CSD LUT for each multiplier coefficient constituent in the interpolation digital 
subfilters. For FRM IIR digital filters, the multiplier coefficients correspond to the bilinear-LDI 
allpass digital networks Gg(z) and G4 (z). 

e A particle with By coordinates is formed in which each coordinate serves as an index of the 
corresponding CSD LUT for each multiplier coefficient in the FIR masking digital subfilters 
Fo(z) and Fy (z). 


. Generation of Initial Swarm: An initial swarm of K particles is formed by generating a random 
cloud around the seed particle as discussed in section 8.1. 


. Fitness Evaluation: The fitness function for CSD FRM IIR digital filters is defined in accordance 
with 


fitness magnitude = —20log[max(ep, £a)] (62) 
fitness group—delay = Çp (63) 
fitness = fintesSimagnitude a FfitNeSS group—delay (64) 
where 
_ joy 

£p = max [Wp|H(el) — 1]] (65) 

wEAwy 
£a = max [W,y|H(e)|] (66) 

weAw, 
Gp = max [Weglt(w) — prl] (67) 


wEAWwy 
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with Awp representing the passband frequency region(s), with Aw, representing the stopband 
frequency region(s), and with T(w) representing the group-delay frequency response of the FRM 
IIR digital filter. Here, Wp, Wa, and Wed represent weighting factors for the passband and stopband 
magnitude responses, and for the group-delay response, respectively. Moreover, 4r represents the 
average group-delay over the passband region. 

In [48], a convenient way to represent digital networks in terms of matrix representation is presented. 
This technique can be used to find the magnitude and group delay frequency response of the digital 
network in Fig. 12. Let us consider the input to the digital network in Fig. 12 to be xp and the 
output of it to be yp. In addition, let the output of the i-th time delay in Fig. 12 to be x; and the 
input to the i-th time delay to be y;. The transfer function matrix of the network, T, can be found as 


y =x (68) 


where y = [YD, Y1, Y2,- --,Y2m41]f Zand x = [xp, x1, X2,- - -, X2m41]f, and T is a (2m +2) x 
(2m + 2) matrix with the entries obtained as Eqn. (69). 


0 1 =1 0 0 D 0 0 
1, 0 0 0 0 OF a 0 0 
m 
mc, mc 1- mc, (1+ D mr, ) =mc mcm Mc +. Mey Min -mc 
i=1 
0 0 mL, 1 0 eee 0 0 
T= 0 0 mc, ML 0 onc, m, Moy + 0 0 (69) 
0 D ML, 0 =m, 1 sii 0 0 
0 0 MC Ln 0 0 0 f ncn Mim MCm 
0 0 MLm 0 0 0 =M 1 
Since x; = z~ yj, the transfer function G(z) = 2 can be found as 


G(z) = z eI -z7!D] te (70) 


where e is a row vector and c is a column vector of length 2m + 1, and where I is the identity 
matrix and D is a (2m + 1) x (2m + 1) matrix in accordance with 


T= (71) 


2 X! denotes the transpose of the matrix X. 
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The matrix T is also useful in finding the group delay of H(z). The group-delay of H(e/”) is given 
by 


7 1 dH(e/”) 
T(w) = ee (72) 


With the help of Eqn. (12), the expression ane can be written as 


E ratev) + Fe) + 
d(Fo(e”) + ie”) 
dw 
= (Fole) — Fy(el”)) + 
d(Fo(e”) — Fi (e9 )) 
dw 


Go(el)+ 


G(e)| (73) 


The derivative of FIR filters can be easily found from their transfer function. In order to find the 
derivative of the digital allpass networks Go(z) and G4 (z), the following expression can be used 


dG el i ant iw jw 
ED) = jio S Guile) Giy() 74) 
i=1 


where G,;(Z) is the transfer function between xp and y;, and where Gj,(z) is the transfer function 
between x; and yp. The transfer functions G,;(z) and Gj,(z) can be found from the transfer 
function matrix T as follows 


Gui(z) = axi +2 'exi[I- 2D] “e (75) 
Giy(Z) = diy + z tell — 2D tci (76) 
where Ayi and Ajy are scalars, ey; is a row vector and cj, is a column vector of length 2m + 1, in 
accordance with [a,; exi] is the i-th row of the matrix T, and [a;, cyl" is the i-th column of 
the matrix T. Having the expressions for H (ei) and ane the group delay can be obtained in 
accordance with Eqn. (72). 

The passband and stopband weighting factors Wp and W; are easily determined from user 
specifications. The group-delay weighting factor is set as 


= Çx fitness magnitude 


JitneSSeroup—delay 


Wed (77) 


where ¢ is a fixed constant such that 0 < ¢ < 1, and where fitnesSinggnitude and fitNeSS group—delay 
are obtained by examining the seed FRM digital filter particle. The weighting factor for the 
group-delay increases as € — 1. 
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Maximum Passband Ripple Ap 0.1[dB] 
Minimum Stopband Loss A, 40[dB] 


Lower Stopband-Edge Normalized Frequency wa, 0.31 7t[Rad] 


Lower Passband-Edge Normalized Frequency wp, _0.337t[Rad] 


Upper Passband-Edge Normalized Frequency wp, _0.607t[Rad] 
Upper Stopband-Edge Normalized Frequency wa, 0.617r[Rad] 


Normalized Sampling Period T 1[s] 
Lowpass Filter Interpolation Factor M; 6 
Highpass Filter Interpolation Factor Mnp 5 


Table 3. Design Specifications for Bandpass FRM IIR Digital Filter 


700 04 2 2 =5 5 10 10 


Table 4. PSO Design Parameters for Bandpass FRM IIR Digital Filter 


Lo lb f L h f 
11 3 #10 12 3 7 


Table 5. CSD Parameters for Bandpass FRM IIR Digital Filter 


10. Application examples 


10.1. Bandpass FRM IIR digital filter design example 


Consider the design of a bandpass FRM IIR digital filter satisfying the magnitude response design 
specifications given in Table 3 over the CSD multiplier coefficient space. 


The parameters for the PSO of bandpass FRM IIR digital filter is shown in Table 4 and the CSD 
parameters are presented in Table 5. 


Given the design specification in Table 3, The order of the digital allpass networks Go,, (z), Gy 7 (z), 
Gorp (z) and G1 ie (z) are found to be 3, 4, 3 and 4, respectively. In addition, the digital masking subfilters 
Fop (2); Fi, (z), Fo,,,(z) and Fi; (z) have the same length as the previous example, i.e. 24, 42, 25 and 
35 respectively, resulting in N = 140. In this example a set of fifteen CSD LUTs are required, fourteen 
LUTs for the multiplier coefficients MCa MCoz MCy3> ML» MLy3> MC Mhra MC, and ML» 
constituent in the digital allpass networks Go,, (z), Gi, (z), Gon, (Z) and Gy,,, (z), and one template 
LUT for all the multiplier coefficients constituent in the masking digital subfilters Fo, (2). Fip (z), 
Fo, (z) and Fy, (z). 


Finally, by using Parks McClellan approach, the subfilters Fo, (z), Fip (z), Fo, (z) and F,,, (2) can be 
designed. Also, by using the EMQF technique, the digital allpass networks Go,, (z), Gi, (z), Gorp (z) 
and G1 hp (z) can be designed. Consequently, the magnitude and group delay frequency responses of the 
overall infinite-precision bandpass FRM IIR digital filter H(z) is obtained as shown in Figs. 13 and 14. 
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Figure 13. Magnitude Frequency-Response of the Overall Figure 14. Group Delay Frequency-Response of the Overall 
Infinite-Precision Bandpass FRM IIR Digital Filter H(e/“’) Infinite-Precision Bandpass FRM IIR Digital Filter H(e/’) 
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Figure 15. Magnitude Frequency-Response of the Overall Figure 16. Group Delay Frequency-Response of the Overall 
CSD Bandpass FRM IIR Digital Filter H(e!’) Before PSO CSD Bandpass FRM IIR Digital Filter H (e’®) Before PSO 


Multiplier CSD Representation Decimal Value 


Meji 00001.0001001 0.9297 
MCy> 00010.0001010 1.9219 
Mo» 00000.1000010 0.5156 
MCa 00001.0010010 0.8594 
MC, > 10000.1010000 15.375 
ML), 00001.0001010 0.9219 
ML, > 00000.0010101 0.0859 


Table 6. Digital Multiplier Values for the Lowpass Section of the Bandpass FRM IIR Digital Filter 


Based on the infinite-precision bandpass FRM IIR digital filter, the corresponding CSD FRM IIR initial 


digital filter is obtained to have a magnitude and group delay frequency responses as shown in Figs. 15 
and 16. 


By applying the proposed PSO to the initial FRM IIR digital filter and after about 160 iterations, the 
PSO converges to the optimal bandpass FRM IIR digital filter having a magnitude frequency response 
as shown in Fig. 17. In addition, Fig. 18 gives us a closer look to the magnitude frequency response of 
the passband region of the bandpass FRM IIR digital filter. Fig. 19 illustrates the group delay frequency 
response of the optimized bandpass FRM IIR digital filter. The values of the multiplier coefficients for 
the lowpass and highpass sections of the bandpass FRM IIR digital filter are obtained as summarized in 
Tables 6 and 7. 
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IH(e")] [dB] 


IH(e")| [4B] 
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Figure 17. Magnitude Frequency-Response of the Overall 
CSD Bandpass FRM IIR Digital Filter H(e!) After PSO 
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Figure 18. Magnitude Frequency-Response of the 
Passband Region of the Overall CSD Bandpass FRM IIR 
Digital Filter H(e’) After PSO 
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Figure 19. Group Delay Frequency-Response of the Overall CSD Bandpass FRM IIR Digital Filter H (e/®) After PSO 


Multiplier CSD Representation Decimal Value 


mc, 00001.0010100 0.8438 
MC, 00010.0001001 2.0547 
TL 00000.1000001 0.4922 
mc, 00001.0100010 0.7656 
MC, 10000.0100001 16.2578 
mg, 00001.0000101 0.9766 
mL, » 00000.0010101 0.0859 


Table 7. Digital Multiplier Values for the Highpass Section of the Bandpass FRM IIR Digital Filter 


~Frequency-Response Characteristic Before PSO After PSO __ 

— Maximum Passband Ripple Ap 0.8982[dB] 0.0978[dB] _ 
Minimum Stopband Loss A, 9.1715[dB] 40.0172[dB ] 
Maximum Group Delay 312[Samples] 239[Samples] 


Table 8. Frequency-Response Analysis of the CSD Bandpass FRM IIR Digital Filter Before and After PSO 


Table 8 represents the comparison of the CSD bandpass FRM IIR digital filters before and after PSO. 
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1. Introduction 


The field of two-dimensional filters and their design methods has known a large development 
due to its importance in image processing (Lim, 1990; Lu & Antoniou, 1992). There are methods 
based on numerical optimization and also analytical methods relying on 1D prototypes. A 
commonly-used design technique for 2D filters is to start from a specified 1D prototype filter 
and transform its transfer function using various frequency mappings in order to obtain a 2D 
filter with a desired frequency response. These are essentially spectral transformations from s 
to z plane, followed by z to (z,, z,) mappings, approached in early papers (Chakrabarti & Mitra, 
1977; Hirano & Aggarwal, 1978; Harn & Shenoi, 1986; Nie & Unbehauen, 1989). Generally these 
transformations conserve stability, so from 1D prototypes various stable recursive 2D filters 
can be obtained. The most common types are directional, fan-shaped, diamond-shaped and 
circular filters. Diamond filters are commonly used as anti-aliasing filters in the conversion 
between signals sampled on the rectangular sampling grid and the quincunx sampling grid. 
Various design methods for diamond-shaped filters were studied in (Tosic, 1997; Lim & Low, 
1997; Low & Lim, 1998; Ito, 2010; Matei, 2010). 


There are several classes of filters with orientation-selective frequency response, useful in tasks 
like edge detection, motion analysis, texture segmentation etc. Some relevant papers on 
directional filters and their applications are (Danielsson, 1980; Paplinski, 1998; Austvoll, 
2000). An important class of orientation-selective filters are steerable filters, synthesized as a 
linear combination of a set of basis filters (Freeman & Adelson, 1991) and steerable wedge 
filters (Simoncelli & Farid, 1996). A directional filter bank (DFB) for image decomposition in 
the frequency domain was proposed in (Bamberger, 1992). In (Qunshan & Swamy, 1994) 
various 2D recursive filters are approached. Fan-shaped, also known as wedge-shaped filters 
find interesting applications. Design methods for IIR and FIR fan filters are presented in some 
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early papers (Kayran & King, 1983; Ansari, 1987). An efficient design method for recursive fan 
filters is presented in (Zhu & Zhenya, 1990). An implementation of recursive fan filters using 
all-pass sections is given in (Zhu & Nakamura, 1996). In (Mollova, 1997), an analytical least- 
squares technique for FIR filters, in particular fan-type, is proposed. Design methods for 
efficient 2D FIR filters were treated in papers like (Zhu et al., 1999; Zhu et al., 2006). Zero-phase 
filters were studied as well (Psarakis, 1990). Different types of 2D filters derived from 1D 
prototypes through spectral transformations were treated in (Matei, 201 1a). 


We propose in this chapter some new design procedures for particular classes of 2D filters; the 
described methods are mainly analytical but also include numerical approximations. Various 
types of 2D filters will be approached, both recursive (IIR) and non-recursive (FIR). The design 
methods will focus however on recursive filters, since they are the most efficient. 


The proposed design methods start from either digital or analog 1D prototypes with a desired 
characteristic. In this chapter we will mainly use analog prototypes, since the design turns out 
to be simpler and the 2D filters result of lower complexity. This analog prototype filter is 
described by a transfer function in the complex variable s, which can be factorized as a product 
of elementary functions of first or second order. The prototype transfer function results from 
an usual approximation (Butterworth, Chebyshev, elliptic) and the shape of the frequency 
response corresponds to the desired characteristic of the 2D filter. 


The next design stage consists in finding the specific complex frequency transformation from 
the axis s to the complex plane (z,, z), of the general form F:C—C?, s > F(z} z»). This 
mapping will be determined for each type of 2D filter separately, starting from the geometrical 
specification of its shape in the frequency plane. Once found this particular mapping, the 2D 
filter function results directly by applying this transformation to each factor function of the 
prototype. Thus, the 2D filter transfer function H (z,, z2) results directly factorized, which is a 
major advantage in its implementation. The proposed design method applies the bilinear 
transform as an intermediate step in determining the 1D to 2D frequency mapping. All the 
proposed design techniques are mainly analytical but also involve numerical optimization, in 
particular rational approximations (e.g. Chebyshev-Padé). Some of the designed 2D filters 
result with complex coefficients. This should not be a serious shortcoming, since such IIR filters 
are also used (Nikolova et al., 2011). 


In this chapter we will approach two main classes of 2D filters. The first one comprises three 
types of orientation-selective filters, as follows: square-shaped (diamond-type) IIR filters, with 
arbitrary orientation in the frequency plane; fan-type IIR filters with specified orientation and 
aperture angles; and very selective IR multi-directional filters (in particular two-directional 
and three-directional), which are useful in detecting and extracting simultaneously lines with 
different orientations from an image. 


The other class discussed here refers to FIR filters. From this category we will approach zero- 
phase filters with circular frequency response. Zero-phase filters, with real transfer functions, 
are often used in image processing since they do not introduce any phase distortions. All these 
types of 2D filters are analyzed in detail in the following sections. 
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Stability of the two-dimensional recursive filters is also an important issue and is much more 
complicated than for 1D filters. For 2D filters, in general, it is quite difficult to take stability 
constraints into account during approximation stage (O’Connor & Huang, 1978). Therefore, 
various techniques were developed to separate stability from approximation. If the designed 
filter becomes unstable, some stabilization procedures are needed (Jury et al., 1977). Various 
stability conditions for 2D filters have been found (Mastorakis, 2000). 


The medical image processing field has known a rapid development due to imaging value in 
assisting and assessing clinical diagnosis (Semmlow, 2004; Berry, 2007; Dougherty, 2011). In 
particular, the currently used vascular imaging technique is x-ray angiography, mainly in 
diagnosing cardio-vascular pathologies. A frequent application of cardiac imaging is the 
localization of narrowed or blocked coronary arteries. Fluorescein angiography is the best 
technique to view the retinal circulation and is useful for diagnosing retinal or optic nerve 
condition and assessing disorders like diabetic retinopathy, macular degeneration, retinal vein 
occlusions etc. There are many papers approaching various methods and techniques aiming 
at improving angiogram images. In papers like (Frangi et al., 1998) the multiscale analysis is 
used, with the purpose of vessel enhancement and detection. Usual approaches include 
Hessian-based filtering, based on the multiscale local structure of an image and directional 
features of vessels (Truc et al., 2007). In cardio-vascular imaging, an essential pre-processing 
task is the enhancement of coronary arterial tree, commonly using gradient or other local 
operators. In (Khan et al., 2004) a decimation-free directional filter bank is used. An adaptive 
vessel detection scheme is proposed in (Wu et al., 2006) based on Gabor filter response. 
Filtering is an elementary operation in low level computer vision and a pre-processing stage 
in many biomedical image processing applications. Some edge-preserving filtering techniques 
for biomedical image smoothing have been proposed (Rydell et al., 2008; Wong et al., 2004). 
At the end of this chapter some simulation results are given for biomedical image filtering 
using some of the proposed 2D filters, namely the directional narrow fan-filter with specified 
orientation and the zero-phase circular filter. 


2. Analog and digital 1D prototype filters used in 2D filter design 


This section presents the types of analog and digital 1D recursive prototype filters which will 
be further used to derive the desired 2D filter characteristics. An analog IIR prototype filter of 
order N has a transfer function in variable s of the general form: 


_ PS) Sy. Sas 
Hpl)= oy = LP [Xa s (1) 


This general transfer function can be factorized into simpler rational functions of first and 
second order. Such a second-order rational function (biquad) can be written: 
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H,(s)=K(s* +b,s +09) /(s? + a,5 +a) (2) 


where generally the second-order polynomials at numerator and denominator have complex- 
conjugated roots, and k is a constant. For typical approximations — Chebyshev or elliptic — 
usually b, =0, therefore the numerator has imaginary zeros. For odd-order filters, the denom- 
inator contains at least a first-order factor (s +a). An elliptic approximation with very low 
ripple can be used for an almost maximally-flat low-order filter. Next we consider two such 
low-pass (LP) prototypes with imposed specifications. The first is an elliptic LP analog filter 
of order N =6, cutoff frequency w,=0.47, peak-to-peak ripple R,=0.04dB, stop-band attenua- 
tion R,=38dB. Its transfer function can be factorized into three biquad functions like (2): 
Hp(s)=k - H (s) Hy9(s) > H,3(8) where k =2.375 and: 


H,,(s) =(s? + 39.195)/(s? +0.2221s + 2.8797) (3) 
Hyp(s) = (s? + 6.5057)/(s? + 0.9172s + 2.4291) (4) 
Hy3(s) = (s? + 4.2217) /(s? + 2.0448s + 1.5454) (5) 


The frequency response magnitude of this LP filter for w €[-7, 71] is shown in Fig. 1(a). 


The second prototype is an elliptic LP analog filter with parameters: N =4, w,=0.47, R,=0.05 
db,R,=36db. Its transfer function is written as a product of two biquad functions like (2): 
H(s)=k - H,,(s) - H,,(s), where k = 0.01 and 


H,,(s) = (s? +33.385)/(s? + 0.5894s + 2.2398) (6) 


H, (8) = (8? + 6.4226)/(s? +1.9691s + 1.5266) (7) 


The frequency response magnitude of this LP filter for w E[- 7, 7] is shown in Fig.1 (b). The 
simplest analog LP filter has a transfer function H j(s)=a /(s +a), where the value a gives the 


selectivity (Fig.1(c)). If the filter characteristic is shifted to a given frequency wy, €[-7, 7], the 


transfer function becomes: 


H,s(s) =a/(s +a + j-@)) (8) 
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In Fig.1 (d) the shifted filter response magnitude for wp,=0.4167 is shown. Another useful 


analog prototype is the selective second-order (resonant) filter with central frequency wy: 
H,(s)=as/(s? +as+ o) (9) 


The transfer function magnitude for such a filter with a=0.1 and wy=1.3 is shown in Fig. 1 


(e). This will be further used as a prototype for two-directional filters. 


3 2 1 ] 1 2 3 a 2 1 


© (a) (e) 


Figure 1. Frequency response magnitudes of: (a) LP elliptic prototype of order 6; (b) LP elliptic prototype of order 4; 
very selective first-order filter with central frequencies w )=0 (c) and wọ=0.4167 (d); (e) selective band-pass filter with 


Wy = 1.3 


A useful zero-phase prototype can be obtained from the general function (1) by preserving 
only the magnitude characteristics of the 1D filter; this prototype will be further used to design 
2D zero-phase FIR filters of different types, specifically circular filters, with real-valued 
transfer functions. In order to obtain a zero-phase filter, we consider the magnitude charac- 
teristics of H (jw), defined by the absolute value | Hp(jw) |=! P(jw)!/ 1 Q(jw)!. We look 
for a series expansion of the magnitude | H,(jw) | that has to be an approximation as accurate 


as possible on the frequency domain [-7, 7]. The most convenient for our purpose is the 
Chebyshev series expansion, because it yields an efficient approximation of a given function, 
which is uniform along the desired interval. The Chebyshev series in powers of the frequency 
variable w for a given function on a specified interval can be easily found using a symbolic 
computation software like MAPLE. However, we will finally need a trigonometric expansion 
of |H p(jæ) l, namely in cos(nw), rather than a polynomial expansion in powers of w. 


Therefore, prior to Chebyshev series calculation, we apply the change of variable: 
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@ =arccos(x) = x = cos(@) (10) 


and so we get the polynomial expansion in variable x: 


|Hp(arccos(x) l= Èa, "x" =a) tax +a, x? +a? +. Haya (11) 


n=0 


where the number of terms N is chosen large enough to ensure the desired precision. The next 
step is to substitute back x=cosw in the polynomial expression (11), therefore we obtain the 
factorized function in cosw, with n +2m=N: 


m 


meTag cos”(@) =k- [[(coso+a,) -[T] (cos? @ +, ;COSM +A) (12) 
n=0 i=1 j=l 


Next let us consider a recursive digital filter of order N with the transfer function: 


_ P(z) N yl 
p(2) 00) =p JÈ z (13) 


This general transfer function with M =N can be factorized into first and second order rational 
functions. For an odd order filter, H p(z) has at least one first-order factor: 


H,(z) =(b,z + by) /(z +a) (14) 


The transfer function also contains second-order (biquad) functions, where in general the 
numerator and denominator polynomials have complex-conjugated roots: 


H, (2) =(b,2” +b,z+by) {(2? +02 +a) (15) 
We will further use the term template, common in the field of cellular neural networks, for the 
coefficient matrices of the numerator and denominator of a 2D transfer function H (z4, Z,). 
3. Diamond-type recursive filters 


In this section a design method is proposed for 2D square-shaped (diamond-type) IIR filters. 
The design relies on an analog 1D maximally-flat low-pass prototype filter. To this filter a 
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frequency transformation is applied, which yields a 2D filter with the desired square shape in 
the frequency plane. The proposed method combines the analytical approach with numerical 
approximations. 


3.1. Specification of diamond-type filters in the frequency plane 


The standard diamond filter has the shape in the frequency plane as shown in Fig.2 (a). It is a 
square with a side length of n42, while its axis is tilted by an angle of p=7 /4 radians about 
the two frequency axes. Next we will consider the orientation angle p about the w,- (vertical) 
axis. In this chapter a more general case is approached, i.e. a 2D diamond-type filter with a 
square shape in the frequency plane, but with arbitrary axis orientation angle, as shown in 
Fig. 2(e). Next we refer to them as diamond-type filters, since they are more general than the 
diamond filter from Fig. 2 (a). 


The diamond-type filter in Fig.2 (e) is derived as the intersection of two oriented low-pass 
filters whose axes are perpendicular to each other, for which the shape in the frequency plane 
is given in Fig.2 (c), (d). Correspondingly, the diamond-type filter transfer function H p(2,, z3) 


results as a product of two partial transfer functions: 


Fp (21,29) = Ay (21,29) Hy (21,29) (16) 


®© 


Figure 2. (a) diamond filter; (b) wide-band oriented filter; (c), (d) wide-band oriented filters with orientations forming 
an angle g=z7t / 2; (e) square-shaped filter resulted as product of the above oriented filters; (f) rhomboidal filter 


The frequency characteristic of H,(z,, z») is ideally identical to the frequency characteristic of 
H(z; Z) rotated by an angle of p=7/2. Since this rotation of axes implies the frequency 
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variable change: w >w», w> -w the transfer function H,(z,, z,) can be derived from 
H(z; 29) as H,(Z, 2) = Hy (Zy zo): A more general filter belonging to this class is a rhomboi- 


dal filter, as shown in Fig.2 (f). In this case the two oriented LP filters may have different 
bandwidths and their axes are no longer perpendicular to each other. 


3.2. Design method for diamond-type filters 


The issue of this section is to find the transfer function H,,,(2Z,, z2) of the desired 2D filter using 
a complex frequency transformation s > F(z,, z,). From a prototype Hp(s)=Hp( jw) (which 
varies on one axis only), a 2D oriented filter is obtained by rotating the axes of the plane (w,, wy) 
by an angle @. The rotation is defined by the following linear transformation, where w,, w, are 
the original frequency variables and w,, w, the rotated ones: 


@ |_| coso sing| | a, 
el ee | H Ha) 


The spatial orientation is specified by an angle p with respect to œw-axis, defined by the 1D 
to 2D frequency mapping w > w,cos~ + w sinp. By substitution, we obtain the oriented filter 
transfer function H(@,, w,)=Hp(wcosp + w,sing). In the complex plane (s4, s3) the above 


frequency transformation becomes: 
Ss, COS Ọ + S, Sing (18) 


The oriented filter H,(w,, w,) has the frequency response magnitude section along the line 
w,cose + w,sing =0, identical with prototype H p(w), and constant along the perpendicular line 
(filter longitudinal axis) w,sing -w,cosp =0. The usual method to obtain a discrete filter from 
an analog prototype is the bilinear transform. If the sample interval takes the value T =1, the 
bilinear transform for s, and s, in the complex plane (s4, s,) has the form: 


8, =2(z, -1)/(z, +1) s, =2(z, -1)/(z, +1) (19) 


This method is straightforward, still the resulted 2D filter will present linearity distortions in 
its shape, which increase towards the limits of the frequency plane as compared to the ideal 
frequency response. This is mainly due to the so-called frequency warping effect of the bilinear 
transform, expressed by the continuous to discrete frequency mapping: 


o =(2/T)-arctg(o,T/2) (20) 
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where w is a frequency of the discrete filter and w, is the corresponding frequency of the analog 
filter. This error can be corrected by applying a pre-warping. Taking T =1 in (20), we substitute 
the mappings: 


a, >2 -arctg(@,/2) 0, > 2-arctg(@,/2) (21) 


In order to include the nonlinear mappings (21) into the frequency transformation, a rational 
approximation is needed. One of the most efficient is Chebyshev-Padé, which gives uniform 
approximation over a specified range. We get the accurate approximation for w €[-7, 7]: 


arctg (/2) = 0.4751: @/(1+0.05-a*) (22) 


Substituting the nonlinear mappings (21) with approximate expression (22) into (18) we get 
the 1D to 2D mapping which includes the pre-warping along both frequency axes: 


$COSP s sino 
1-0.05-s? 1-0.05-s; 


s > F (s153) = 0.95- (23) 


Applying the bilinear transform (19) along the two axes we obtain the mapping s > F(z, z2) 


in matrix form, where z,=[1 z; z] and z,=[1 z, ae): 


s> F (212) =k: M (21/2) /N g( 21,22) =k- (z1 XM, x z3) /(z1 XN, x23) (24) 


Here k =1.5233 and the matrices M, and N, of size 3 x3 are given by: 


St i A O it 8a 
M,=cosg-/ 0 0 O|tsing-/-3 0 3|N,=|3 9 3 (25) 
i 4 4 10i 131 


Substituting the mapping (24) into the expression (2) of the biquad transfer function H,(s) with 
b,=0, we get the 2D transfer function H ,(2,, Z,) in the matrix form: 


Hoy(21/29) = (24 xB, x23 }/(z, x A, x23 | (26) 


where z4 and z, are the vectors: 
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2 3 „4 Baa gi 
z,=|1 Ze 2 2i a] zeli 2o 27. 25 A (27) 
and the 5x5 templates B,, A, are given by the expressions: 
2 : _ 42 
B, =k "M, *M, +b: N *N, ; A; =k MFM +4: kM, FN +0: N *N, (28) 


For instance, corresponding to the third biquad function H,,(s) given by (5), the following 5x5 


templates result according to the expressions (28): 


(0.2464 0.9407 1.1418 0.4027 0.0671 
0.9407 3.2233 3.8917 1.6092 0.4027 
B,=|1.1418 3.8917 6.1484 3.8917 1.1418 
0.4027 1.6092 3.8917 3.2233 0.9407 
0.0671 0.4027 1.1418 0.9407 0.2464 
r 0.0947 0.1941 0.0732 -0.0163 0.0245 
0.1941 -0.2738 -0.7181 0.0774 0.3112 
A,=| 0.0732 -0.7181 1.0000 2.8851 1.2743 
-0.0163 0.0774 2.8851 3.6570 1.1768 
~ 0.0245 0.3112 1.2743 1.1768 0.3131 


Figure 3. (a) LP correction filter characteristic; frequency response magnitudes and contour plots of: (b), (c) uncorrect- 
ed diamond-type filter; (d), (e) corrected diamond-type filter 
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The characteristics of a diamond-type filter with orientation angle p=7/4 and based on the 
prototype filter of order 6 given by the factors (3)-(5) is shown in Fig.3 (b), (c). As can be noticed, 
the filter characteristic corrected by pre-warping has a good linearity, however it still twists 
towards the margins of the frequency plane. These marginal linearity distortions can be 
corrected using an additional LP filter. For instance, we can choose as prototype an 1D elliptic 
digital filter of order N =3, pass-band ripple R,=0.1dB, stop-band attenuation R,=40 dB and 


cutoff frequency w,=0.6, which has the coefficients given by the vectors: 
Bo =[0.3513 1.01 1.01 0.3513] Ac, =[1 0.9644 0.6701 0.088] (29) 


The 2D low-pass filter is separable and results by applying successively the 1D filter along the 
two frequency axes; the 4x4 matrices of the correction filter result as: B= Be, 18 Boy 
Ac= Acı ® Acı where the symbol ® denotes outer product of vectors. The correction filter has 


the following transfer function, where z,=[1 2, z z], z=[1 z zi x : 


Holgy) = (24 » Be x23) /(z, Ac 2] (30) 


Figure 4. Diamond-type filters with orientation angle: (a), (b) =t / 12; (c), (d) p=7/6 


The resulted 2D square-shaped correction filter characteristic is shown in Fig.3 (a) and is al- 
most maximally-flat, as required. The corrected version of the diamond-type filter from Fig. 
3 (b), (c) has the magnitude and the contour plot shown in Fig. 3 (d), (e). It can be easily no- 
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ticed that the initial distortions have been eliminated. Another two diamond-type filters 
with orientation angles p=7 /12 and p=7/6 are shown in Fig. 4 (a)-(d). 


4, Fan-type recursive filters 


In this section an analytical design method in the frequency domain for 2D fan-type filters is 
proposed, starting from an 1D analog prototype filter, with a transfer function decomposed 
as a product of elementary functions. Since we envisage designing efficient 2D filters, of 
minimum order, recursive filters are used as prototypes, and the 2D fan-type filters will result 
recursive as well. 


In Fig.5 (a) a general fan-type filter is shown, with an aperture angle < BOD =0, oriented along 
an axis CC ' and its longitudinal axis forming an angle < AOC =ọ with frequency axis Ow,. A 
particular case is the two-quadrant fan filter, shown in Fig.5 (b). Fig.5 (c) shows a DFB with 8- 
band frequency partition (Bamberger, 1992), an angularly-oriented image decomposition 
which splits the frequency plane into fan-shaped sub-bands (channels). 


Figure 5. (a) Ideal fan filter with given aperture, oriented at an angle g; (b) Ideal two-quadrant fan filter (c) 8-band 
partitions of the frequency plane 


The 1D analog filter discussed in section 2 is used as prototype. The general fan-type filter can 
be derived from a LP prototype using the frequency mapping (Matei & Matei, 2012): 


o> f,(@,@,) =a (o, ‘COSO - O, -sing) /(o, -sing +0, cose) (31) 


In (31), a=1/tg(@/2) is the aperture coefficient, where 0 is the aperture angle of the fan-type 
filter. This frequency mapping in the complex variables s,= jw,, $)= j@ is: 


S> fp(S152) =j-a-(s, “COSP—S, -sing)/(s, ‘sINQ+S, -cos@) (32) 
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Figure 6. Frequency response magnitudes and contour plots for: (a) fan-type filter with aperture @=0.1 7t and orienta- 
tion g=7/7; corrected filters with @=0.1 77, p=7/7 (b) and p=0(c) 


Applying the same steps as in Section 3.2 in order to obtain a discrete form of the above 


frequency mapping, using relations (21), (22) and (32) we obtain the 1D to 2D mapping which 
includes pre-warping along both axes of the frequency plane: 


(s\(1-0.05s3)cos ~s,(1-0.05s;)sing) 


s>F,(s,,8))=j-a- (33) 


(s,(1- 0.0583) sing +s,(1-0.05s;)cos¢) 


We now apply the bilinear transform (19) along the two axes and obtain the mapping 


s>F pEr Z) in matrix form, where z,=[1 2, 2] and z,=[1 z, z7]: 


8 > Fyn) a P u) aP z) a0 a) (34) 


and the 3x3 matrices P, and Q, are given by: 


-1 -3 -1 -1 0 1 -1 -3 -1 -1 0 1 
P, =cos@: 0 0 0ļ-sinọ-|-3 0 3 Qo =sing- 0 0 Oj}+cosg:}-3 0 3 (35) 
1 3 1 -1 0 1 1 3 1 -1 0 1 


Substituting the mapping (34) into the biquad expression (2) with b4 =0, we get the 2D transfer 
function in matrix form Hy 4(Z, z)=(z, x B,x zi) / (zx Ax z), similar to (26), where the 


vectors Z,, Z, are given by (27). The 5x5 templates B, and A, are given by: 
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B, =b)-Q,*Q,-a°-P,*P,; A,=4a,-Q,*Q,-a@ -P,*P,+j-a-a,-P,*Q, (36) 


The 2D transfer function for each biquad is complex. The characteristics of a fan-type filter 
designed with this method and based on the prototype filter of order 4 given by (6)-(7) is shown 
in Fig.6 (a), for the indicated parameters. As with the diamond-type filter analyzed in the 
previous section, the fan-type filter characteristic features marginal linearity distortions which 
can be corrected using a LP filter, similar with the correction filter used in Section 3.2 and 
having the frequency characteristic shown in Fig. 3 (a). 


Figure 7. (a) Frequency response magnitude and (b) contour plot for the 2-quadrant fan filter 


Two corrected fan-type filters with specified parameters have the magnitudes and contour 
plots shown in Fig.6 (b), (c). The initial distortions have been eliminated. With the same 
correction filter, we obtain the two-quadrant fan filter, shown in Fig. 7, by setting the aperture 
angle 0=7 /2 and orientation angle p=7 /4. 


5. Very selective multidirectional IIR Filters 


In this section a design method based on spectral transformations is proposed for another class 
of 2D IIR filters, namely multi-directional filters. The design starts from an analog prototype 
with specified parameters. Applying an appropriate frequency transformation to the 1D 
transfer function, the desired 2D filter is directly obtained in a factorized form, like the filters 
designed in the previous sections. For two-directional filters, an example is given of extracting 
lines with two different orientations from a test image. The spectral transformation used in the 
case of multi-directional filters is similar to the one presented in the previous section, derived 
for fan-type filters and given by (34), (35). In this section the design of two-directional and 
three-directional filters with specified orientation is detailed. The method can be easily 
generalized to arbitrary multi-directional filters. 
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Figure 8. Ideal shapes of some directional filters in the frequency plane: (a) two-directional filter; (b) three-directional 
filter; (c) three-band selective filter with w);=0.597t, wọ2 = — 0.3670 


5.1. Two-directional fan-type filters 


A two-directional 2D filter is orientation-selective along two directions in the frequency plane. 
It is based on a selective resonant IIR prototype as given in section 2. Applying the same 
frequency transformation s > F PACE z) derived for fan-type filters and given by (33) to the 


prototype filter (9) we get the 2D two-directional transfer function H,(z,, z») in matrix form 
H(z; 2) = (z x B3 x z7) / (z, x Ag x zi), similar to (26), but the 5x 5 matrices B3, A, now have the 


form: 


B, =a-a-P,*Q, A, =a-a-P,+Q, + ja:P, +P, -0 Q, +Q,] -n 


The denominator matrix A, has complex elements. In Fig.9 (a), the contour plot of the frequency 


response magnitude is shown for a two-directional filter with aperture 0 = 7 /6 and orientation 
p=n /5. As with the previous types of filters, the marginal linearity distortions can be corrected 
using an additional LP square-shaped filter. 


The templates B,;- and Aç of the corrected filter result by convolution: B,.=B,* Bo and 
Agc =A; * Ac. In Fig.9 (b), (c) the frequency response magnitudes and contour plots are 


displayed for the corrected two-directional filter with specified aperture and orientation. The 
initial distortions have been eliminated. 


The second two-directional filter in Fig.9 (d), (e) is a particular case, being oriented along the 
two frequency axes (O= 71/2, p=7/4), therefore can be used to detect simultaneously hori- 
zontal and vertical lines from an image, as shown in the next section. 


5.2. Three-directional fan-type filters 


In order to design a three-directional filter like the one depicted in Fig. 8 (b), we must start 
from an analog three-band selective filter, like the one with frequency response shown in Fig. 
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a 


(a) (e) 


Figure 9. (a) Contour plot of a two-directional filter with @= 7t / 6, p= / 5; frequency response magnitudes and con- 
tour plots of two corrected two-directional filters with parameters: 0= 7/6, p=7/5 (b), (c) and @= 71/2, ọ=7/4 (d), 


(e) 


8 (c). For a three directional filter, the middle peak frequency can always be taken w)=0, and 
the other two on each side at specified values. The prototype transfer function H p(s) in variable 


s will be in this case the sum of three elementary functions: 


B,(s) a a a 
= + ; + 
Ap(S) S+@ S+Q@t+]-@, Stat+j-@, 


Hp(s) = (38) 


The frequency response of a filter of this kind with parameter values a =0.03, w);=0.597 and 
wg = — 0.3671 is shown in Fig. 8 (c). Substituting the mapping (34) into the expression (8) of the 
elementary function H js(S), we get the 2D transfer function H,(z,, z,) in matrix form: 
H (zy 2)=(z,xB,* 2.) /(z,*A,* 2), where z,=[1 z zł}, z,=[1 z z2] and the 3x3 


templates B,,A, are given by: 
B, =a-Q,;A, =a-Q, +j-(a-P, +@-Q,) (39) 


Each of the three elementary terms in (38) corresponds to a pair of 3x3 templates B, and A, 


given by (39). If the three elementary filters are given by the pairs of templates (Bp, A,,), (Bpv 
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A, ) and (By, A), the templates of size 7x7 of the entire three-directional filter will result by 


summing up the convolutions of elementary templates: 
B, =B, * Ay. *Ayg + Ay * Bp * A3 + Ag, * Ap *B,3 7 A; = Api * Ap * Ay; (40) 


The numerator B,(s) of H p(s) from (38) has the general form: 
Bp(s)=@-(a, +s +a,-8-+A9) (41) 


where a)=3, a= 60 + j - 2(wg + wp) aNd a=3a*-Wyy ` Woy + j 2a (Wo + wo). 


We see that a, is real and a, a, are generally complex. The coefficients a, a, are real only when 
Wo= — Woy Le. for symmetric frequency values around the origin. Finally, for any specified set 


of values a, Wg}, Wo) the denominator factorizes as B,(s)=3a - (s + r)\(s + r) wherer andr, are 


complex roots. Therefore the factorized prototype transfer function is: 


_ Bis) | 3a-(s+r,)(st+r,) 
A(s) (st+a)(s+p,)(s+p) 


H (8) (42) 


(a) (b) 


Figure 10. Frequency response magnitude (a) and contour plot (b) of a corrected three-directional filter with parame- 
ters: 0=0.237, p=0.2777, Wo, =0.5977, Wo2= — 0.3670 


At the denominator, we denoted p,=a + j + @o, P2754 + j * wp. Applying to each factor the 
frequency transformation s > F „(Zy z2) given by (34), after some algebraic manipulations, we 


finally obtain the templates of the three-directional filter as discrete convolutions: 
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B, =3a- Q, *(7,Q, 4 jaP,) *(nQ, + jaP,) (43) 


A, = (aQ, $ jaP,) x (rQ, $ jaP,) x (PQ, rs jaP,) (44) 


This implies the fact that the transfer function H3(z,, z2) of the 2D three-directional filter with 
templates B, and A, of size 7x7 results directly in a factorized form, which is an important 
advantage in implementation. As a general remark on the method, using an analog prototype 
instead of a digital one, as is currently done, simplifies the design in this case, as the frequency 
mapping results simpler and leads to a 2D filter of lower complexity. The designed filters result 
withcomplexcoefficients, howeversuchIIRfilterscanalsobeimplemented (Nikolovaetal.,2011). 


6. Directional IIR filters designed in polar coordinates 


We approach here a particular class of 2D filters, namely filters whose frequency response is 
symmetric about the origin and has at the same time an angular periodicity. The contour plots 
of their frequency response, resulted as sections with planes parallel with the frequency plane, 
can be defined as closed curves which can be described in terms of a variable radius which is 
a periodic function of the current angle formed with one of the axes. 


It can be described in polar coordinates by p =p(p), where ọ is the angle formed by the radius 
op with w,-axis, as shown in Fig.8(a) for a four-lobe filter. Therefore p(@) is a periodic function 


of the angle ¢ in the range p €[0, 277]. 


6.1. Spectral transformation for filters designed in polar coordinates 


The main issue approached here is to find the transfer function of the desired 2D filter 
Hp(Z Z2) using appropriate frequency transformations of the form w — F (w, w,). The 


elementary transfer functions (14) and (15) have the complex frequency responses: 


H (ja) = (bo +b, cosa + jb, sina) /(a, +cos@+ jsina) (45) 


; b, +(b, +b,)cos@+ j(b, —b,)sino P 
H, (jo) = 1 ( 2 o) I 2 o) = (a) 


a,+(1+a))cos@+j(l-aj)sino Q(@) (46) 


The proposed design method for these 2D filters is based on the frequency transformation: 


F:R >C? o > F(2,,25) = By (2,2) /Ap(%/29) (47) 
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which maps the real frequency axis w onto the complex plane (z,, z»), defined by the real 


frequency mapping: 
: 2 -2 E 
F,:R-R’,o > F(@,,@) =(0; +03) /p(,,0,) (48) 


In (48) p(w, w) is initially determined in the angle variable p as p(¢~) and can be referred to 
as a radial compressing function. In the frequency plane (w4, wy) we have: 


cose = 0, / Jo +o (49) 


Figure 11. (a) contour plot of a four-lobe filter; (b) periodic function p(@); (c) LP prototype 


If the radial function p(p) can be expressed in the variable cose, using (49) we obtain by 
substitution the function p(w, w). The function p(P) will result as a polynomial or a ratio of 


polynomials in cos@. For instance, the four-lobe filter with the contour plot given in Fig.11 
(a) corresponds to a function: 


p(y) =a+bcos4y =a +b -8bcos? o + 8bcos' o (50) 


plotted in Fig.11 (b) on the range p € [0, 27]. More generally, the 2D filter can be rotated in the 
frequency plane with a specified angle p) about one of the frequency axes, e.g. O -w,. For 
instance, in a four-lobe filter, two opposite lobes are oriented along a direction at an angle 
Po and the other two at po + 7 /2, as shown in Fig.12 (b). It can be shown that the cosine of the 


current angle p with initial phase pọ can be expressed as: 


cos?(p + p) = (cos? P TA +sin* P -07 +0.5sin2Qy - o,0,)|(o + o) (51) 


For filters with an even number of lobes, the radial function p(P) is expressed in even powers 
of cos@ or cos(p + Pp). The frequency transformation (48) can be also expressed as: 
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o> Jo + 0?)/p(@,,0,) = JF,(@,,0) (52) 


In order to obtain a rational expression for the frequency response of the 2D filter from an 
elementary 1D prototype of the form (45) or (46) by applying the frequency mapping (52), we 


need to derive rational expressions for the functions cos¥w and sinyw. Using the Chebyshev- 
Padé method in a symbolic computation software, the following second-order rational 
approximations were found: 


cos Vo = (1.0559 -0.086514 - œ — 0.1304- 0°) /(1 +0.75 - œ — 0.110583- a?) =C,(@)/Ag(@) (53) 


sin Vø = (0.167 + 1.46287 -œ -0.259815 -œ ) /(1+0.75-@-0.110583-@")=S.(@)/Ag(@) (64 


which are sufficiently accurate on the range w € [0,7 ]. Since these functions are developed on 
the range [0,7], their approximations result neither odd nor even. However, using the above 
approximations will lead to relatively complex 2D filters, described by templates of size at 
least 9x9. For the type of filters approached, namely selective two-directional (four-lobe) 


filters, the approximations for cosyw and sinyw need not necessarily hold throughout the 
range [0,7], but only on a smaller range near the origin, corresponding to filter pass-band. 
Using now the Padé method we get the first-order approximations: 


sin Vo = (sọ +5,@)/(1+ ra) cos Vo = (cy +¢,@)/(1+ ra) (55) 


with s)=0.0928, s;=2.5218, cy=1.0104, c,=1.2193, r=1.979, which hold only ona 


narrower range around zero of the interval w € [0,77]. Using (55) instead of (53), (54) will result 
in much more efficient 2D filters, which fully satisfy the imposed specifications. 


We will use here a Chebyshev low-pass second-order filter of the general form (15). For this 
type of filter we have the coefficient symmetry b,=by. According to (46) we can write: 


E b, + 2b, cos Vo _ Po) 
a +(1 +a)cos Vo + j(1-a)sin Vø Q(vo) 


H, (jVo) (56) 


The numerator results real because the imaginary part is cancelled. Substituting the expres- 
sions (55) into this complex frequency response we get the rational approximation: 


H, (ive) B b,(1+ra) + 2b, (cy + c,@) 67) 


a,(1+ra@) +(1+4a9)(cg +60) + J -(1-a9)(Sq +510) 
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which can be also written as: 


= b,(1+1@*) + 2bp (cy +¢,0°) 


H, (jo (58) 


a,(1+1r@) + (1+ ag)(Cy +40?) + j(1-ag)(Sp + 5,07) 


The function (58) has even parity, since it is expressed as a rational function in w°. 


6.2. Two-directional filter design 


We approach now the design of a particular filter type designed in polar coordinates, namely 
two-directional (selective four-lobe) filters along the two plane axes or with a specified 
orientation angle. Let us consider the radial function given by: 


H,(9) =1/(p- Bg) - p +1) (59) 


where B() is a periodic function; let B(p)=cos(4p). We use it to design a 2D filter with four 
narrow lobes in the plane (w,, w). Using trigonometric identities, (59) becomes: 


H, (9) =1/(1+8p-(cosg)? -8p (cosø)*) (60) 


and is plotted for p €[-7, 7] in Fig.12(a). This is a periodic function with period ® =n /4 and 
has the shape of a multi-band (“comb”) filter. In order to control the amplitude of this function, 
we introduce another parameter k, such that the radial function p(y) takes the form 
p(p)=k - H,(p). We get using (49): 


a > F(@,,0) =(@f +(2+8p)o7a; + 03) /(k(@? + 03) (61) 
and the function F,(s,, sy) of the form: 
F, (8/8) = -(st +(2+8p)s7s5 + 33) /(Ks} +85) (62) 


Finally we derive a transfer function of the 2D filter H (z,, z,) in the complex plane (z4, z»). This 
can be achieved if we find a discrete counterpart of the function p(w, w), denoted R(z,, z»). 
A possible method is to express the function p(w, w,) in the complex plane (s4, s,) and then 
find the appropriate mapping to (z,, z2) using the bilinear transform or the Euler approxima- 


tion. Even if generally the bilinear transform is more often used, being a more accurate 
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mapping, especially if a frequency pre-warping is applied to compensate for distortions, for 
the particular type of 2D filter approached here the Euler formula leads to more efficient filters 
and with better characteristics, at the same filter order. We will use the backward Euler method, 
which approximates the spatial derivative 0X /ðx by X[n]-X[n-1], replacing s by 
s=1-z~!. On the two directions of the plane we have: s,=1 -z,1, Sy= 1=z,'. The operators 
s; and s,s, correspond to second-order partial derivatives: 0° /0x*—s?=z,+2z;'-2, 
d /Oy?—s7=z,+z,!-2, 0? /dx0y — 5,85. For the mixed operator s,s,, using repeatedly the 
Euler formula, we get the expression (Matei, 2011 a): 2s,s,=z, +z, + Z, +2) -2-2)2) '—Z1 Z. 


Substituting the above relations into (62) we obtain a frequency mapping similar to (47), with 


the templates: 
0 0 1 0 0 
0 2+8p -8-l6p 2+8p 0 Oo 1 0; );0 10 
B,=|1 -8-l6p 20+32p -8-16p 1/A,;=k-)1 -4 1j|*|1 -4 1/=k-A,*A, (63) 
0 2+8p -8-16p 2+8p 0 0 10|JO0 10 
0 0 1 0 0 


The template A f results as a convolution of two 3x3 matrices. The last step in the design of 


this 2D filter is to apply the frequency transformation (61) to the frequency response (58) and 
we find the filter templates B and A as linear combinations of B f and A fi 


B = (b, + 2b Cy): A; + (rb; + 2b9c,) B, (64) 


A =(a, +(1+a0)co + }(1-a9)50): Ay + (ar +(1+ap)c, + j(1—ap)s,) By (65) 


where b, by, by a, dy are the coefficients of prototype (15). Finally the 2D filter transfer function 


in z, and z, has the following expression, with z, and z, given by (27): 


Hy (24/25) = B(Z,25)/ A(Z,25) (z xBxz3)/(z, x Ax z5 | (66) 


Let us design a two-directional filter following this procedure. As 1D prototype let us consider 
a type-2 low-pass Chebyshev digital filter with the parameter values: orderN =2, stop-band 
attenuation R,=40db and passband-edge frequency w,=0.5 (1.0 is half the sampling frequen- 


cy). The transfer function in z is: 


H (2) = (0.012277 - z? -0.012525 - z + 0.012277) /(z? - 1.850147 - z + 0.862316) (67) 
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For a good directional selectivity we also choose p=30 and k=10. The 2D filter frequency 
response magnitude is displayed in Fig.12 (c) and shows a very good linearity along the two 
directions and practically no distortions in the stop band. The constant level contour in the 
plane (w,, w) is given in Fig.12 (d). Calculating the singular values of filter templates for the 


above parameters we find the vectors S 4, Sp for the templates A, B: 
S, =[2.09347 0.00225 0.0005 0 0| S,=[2.14297 0.01939 0.00387 0 0| 68) 


Taking into account the fact that the first singular value of the templates A and B is much larger 
than the other four, the filter designed above can be approximated by a separable filter. 


The singular value decomposition of a matrix M is written as M =U xS x V where U and V are 
unitary matrices and S is a diagonal matrix containing the singular values. Thus we can 


write for the filter templates A and B: 
A=U, *S,*V,y B= U; x Sg x Vg (69) 


If U rs and V A, are the first columns of the matrices U,, V 4 then A can be approximated by 


a matrix A,=s,,-U,,®V ai where s4; is the largest singular value of A, U Ay and V A, are the 
corresponding columns of U, and V,, ® stands for outer product, T for transposition. 
Similarly for B we find B,=s,,:U,,® Vg. For the specified filter parameters we obtain 
Spi=2.14297 and for template B the column vectors Uz, and Vg, result identical: 
Up, =V p1=(-0.00424 0.3971 -0.82743 0.3971 -0.00424]". 


For the template A we get s ,,;=2.09347 and the vectors U AyY Ay have complex elements. The 


frequency response of the resulted filter is given in Fig.12 (e). As can be noticed, the effect of 
the above approximation is an “overshoot” at zero frequency. This should not affect the filter 
functionality in detecting lines parallel with the two axes. Moreover, since the marginal 
elements of the 5x 1 vectors U 4y V av Ugy Vp, have negligible values, by discarding them we 


obtain the 3x1 vectors: U p= V g)=[0.3971 -0.8274 0.3971)" 
U 4y=[-0.315 0.6334 -0.315]" + j-[0.2573 -0.5175 0.2573)" 
V 49= [0.4067 -0.818 0.4067} — j - [0.0024 -0.0048 0.0024)" 


We finally obtain a very selective two-directional 2D filter implemented with two minimum 
size (3x3) templates. The template B is real while A is complex. The frequency response 
magnitude of this filter is shown in Fig.12 (f) and is practically similar to the one in Fig.12 (e). 
Similarly we can design a two-directional (four-lobe) filter with a specified orientation angle. 
Using the previously described method and based on the Euler approximation, the expression 
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(10) of cos*(~p + pg) corresponds to a frequency transformation in the complex variables z; and 


Zy Written in matrix form as: 


PP) 


(b) 


Hla.) 


© 


Figure 12. (a) Periodic radial function; (b) contour plot of a two-directional filter with orientation @)=7t / 12; (c) fre- 
quency response and (d) contour plot of the two-directional filter with 5x5 templates; frequency response of the filter 
with separable 5x5 (e) and 3x3 templates (f) 


cos?(p +P) > F(z, 22) = By (2/22) Ago(Z-22) -(Z, xB yo x z.) / (Z, xA pX 2: (70) 
0 10 0 00 6 i 4 
By =COS* y|0 -2 O|+sin?g|1 -2 1|+0.25sin(2g)| 1 -1 1 (71) 
0 10 0 00 ai: 4 


and A, is identical to A, from (63). The 5x5 templates of the mapping (47) are given by: 


B A, o * Ago A A, o * Ag + 8p Bog * Ago — 8p Bpo *B yo (72) 


I fo 


The final filter templates result according to relations (64) and (65). 


Regarding the proposed method, the frequency responses of this class of 2D filters can be 
viewed as derived through a radial distortion from a generic maximally-flat circular filter. 
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Indeed, referring to (48), the circular filter is the trivial case for which p(w, w,)=1 and the 


mapping (48) reduces to w° > wf + wf, as expected. This method allows one to design 2D filters 


with anon-convex shape in the frequency plane. The proposed design method does not involve 
global numerical optimization techniques, but only a few numerical approximations. The 
method is more general and can be applied as well to design fan filters, diamond filters, multi- 
directional filters etc. (Matei, 2011 a). 


7. Zero-phase FIR circular filters 
Filters with circular symmetry are very useful in image processing. We propose an efficient 


design technique for 2D circularly-symmetric filters, based on the previous 1D filters, consid- 
ered as prototypes. Given a 1D prototype H p(w), the corresponding 2D circular filter function 


H-(@, w) results using the frequency mapping w > ywr + ws: 
Heloo) = Hp( Jo? + oF (73) 


The currently-used approximation of the circular function cosy; + w; is given by: 


cosa; +0, = C(a,,@,) =-0.5 + 0.5(cosa, + cos @,) + 0.5c0s a, cosa, (74) 
which corresponds to the 3x3 array: 


0.125 0.25 0.125 
C=| 0.25 -0.5 0.25 (75) 
0.125 0.25 0.125 


Let us consider as prototype a LP analog elliptic filter of order N =4, pass-band peak-to-peak 
ripple Rp=0.04 dB, stop-band attenuation R =40 dB and passband-edge frequency Qp=7t /2. 
Its transfer function in variable s is: 


H(s) = 0.1087 -(s* +19.864-s* +84.041) /(s* + 3.2041-s° +8.4315-s* +13.126-s-+14.082) (76) 


Using MAPLE or another symbolic computation program and following the design steps 
described in section 2, we obtain a a polynomial approximation of the magnitude | Hp( ja) | 
through Chebyshev expansion, which has the following factorized form, with x =cosw: 


300 Digital Filters and Signal Processing 


|H_)(@)| = 48.6 - (x + 0.8491)(x + 0.7717)(x — 1.087)(x? + 1.9934x + 0.994) on 
(x? + 1.0797 x + 0.318)(x7 — 0.3849x + 0.1766)(x? —1.2882x + 0.5314)(x? — 1.9338 x + 0.9726) 


In order to obtain a filter with circular symmetry from the factorized 1D prototype function, 
we replace in (12) cosw with the circular cosine function (74). For instance, corresponding to 
(12), the filter template A results in general as the discrete convolution: 


A =k- Aq * Aj) *...* Ay, * Ad) * Ad *...# AD, (78) 


where A; (i=1...1) are 3x3 templates and A,; (j=1...m) are 5x5 templates, given by: 
Ayj=C +4; - Ag and A, ;=C * C +a ;* Co +a j * Ao where Ap, is a 3x3 zero template and Ap a 
5x5 zero template with the central element equal to one; Cy is a 5x5 template 


obtained by bordering C with zeros. The above expressions correspond to the factors in (12). 


The frequency response H((w, w,) of the 2D circular filter results in a factorized form by 
substituting x=C(w,, w) in (77). Even if the filter results of high order, with very large 


templates, next we show that using the Singular Value Decomposition (SVD), the resulted 2D 
filter can be approximated with a negligible error. For the filter template B we can write 
B=U,xS,xV g. The vector of singular values Sp of size 1x27 has 14 non-zero elements: 


Sp, =[0.50536 0.086111 0.032794 0.013627 0.00521 0.002937 0.001935 
0.001061 0.000639 0.000451 0.000418 0.0000385 0.0000196 0.00000144] 


Figure 13. Frequency response magnitude (a) and contour plot (b) of a circular FIR filter 


Let us denote the vector above as Sz,=[s;, |, with k=1...14 in our case. The exact filter matrix B 
can be written as: B=U p,xSp,xV gı where Up, and Vz, are made up of the first 14 columns 
of the unitary matrices Up, and V}. If we consider the first largest M values of the vector 


Sg1=[5;], the matrix B can be approximated as: 
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Figure 14. Frequency response magnitudes and contour plots for the circular filter resulted by taking into account the 
first largest: (a), (b) 8 singular values; (c), (d) 5 singular values 


M 
B = By = X, sp ‘Ug, © Vix (79) 
k=1 


Here B,, is the approximation of matrix B taking into account the first M singular values (in 
our case M <14), while U py Vg, are the k-th columns of the matrices Up, and V gı; ® stands 
for outer product and the superscript T for transposition. 


Fig.14 shows the frequency response magnitudes of the designed circular filter approximated 
by taking into account the first largest 8 singular values and 5 singular values. It can be noticed 
that even retaining only the first 5 singular values, the 2D filter preserves its circular shape 
without large distortions. In this case the filter template B is approximated by Bọ from (79), 
for M=5. Therefore, the template B can be written as a sum of only 5 separable matrices 
according to (79). This is an important aspect in the filter implementation. 


8. Applications and simulation results 


An example of image filtering with a two-directional filter is given. We use the filter shown in 
Fig.3(e), (f). This type of filter can be used in simultaneously detecting perpendicular lines from 
an image. The binary test image in Fig.15 (a) contains straight lines with different orientations 
and lengths, and a few curves. It is known that the spectrum of a straight line is oriented in 
the plane (@, wy) at an angle of 7/2 with respect to the line direction. Depending on filter 
selectivity, only the lines with the spectrum oriented more or less along the filter pass-bands 
will remain in the filtered image. In the output image in Fig.15 (b), the lines roughly oriented 
horizontally and vertically are preserved, while the others are filtered out or appear very 
blurred, due to directional low-pass filtering. The joints of detected lines appear as darker 
pixels and can be detected, if after filtering a proper threshold is applied. 


Let us apply the designed fan-type filters, which can be regarded as components of a DFB, in 
filtering a typical retinal vascular image. Clinicians usually search in angiograms relevant 
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features like number and position of vessels (arteries, capillaries). An angular-oriented filter 
bank may be used in analyzing angiography images by detecting vessels with a given 
orientation. Let us consider the retinal fluorescein angiogram from Fig.16(a), featuring some 
pathological elements which indicate a diabetic retinopathy. This image is filtered using 5 
oriented wedge filters with narrow aperture (0 =r /24), designed using the method described 
in section 4. Fig.16 (b)-(f) show the directionally filtered angiography images. 


(a) 


Figure 15. (a) Test image; (b) filtered image 


(d) 


Figure 16. (a) Retinal fluorescein angiogram; (b)-(f) images resulted as output of five component filters of the fan- 
type filter bank 


As can be easily noticed, the vessels for which the frequency spectrum overlaps more or less 
with the filter characteristic remain visible, while the others are blurred, an effect of the 
directional low-pass filtering (Matei & Matei, 2012). The directional resolution depends on the 
filter angular selectivity given by 0. 


Analytical Design of Two-Dimensional Filters and Applications in Biomedical Image Processing 
http://dx.doi.org/10.5772/52195 


Figure 17. (a) Retinal fluorescein angiogram; (b), (c) results of image filtering with a FIR LP circular filter with cutoff 
frequency Q-,=0.167t and Q-,=0.087 


The designed zero-phase circularly-symmetric FIR filters may be useful as well in pre- 
processing tasks on biomedical images, having a blurring effect on the image which depends 
on its selectivity given by the circular filter bandwidth. The effect is somewhat similar to the 
Gaussian smoothing, which is used as a pre-processing stage in computer vision tasks to 
enhance image structures at different scales. Applying the presented design procedure, a 
circularly-symmetric filter bank can be derived, with components having desired bandwidths. 
Let us consider another retinal fluorescein angiogram, displayed in Fig.17(a). In the simulation 
result shown in Fig.17 (b) and (c), the two circular filters introduce gradual blurring which is 
visible on the fine image details, like small vessels and capillaries. In the image in Fig.17 (c) all 
the finer details have been almost completely smoothed out. 


9. Conclusion 


The design methods presented in this chapter combine the analytical approach based on 1D 
prototype filters and frequency transformations with numerical optimization techniques. For 
the classes of 2D filters designed here, we have used mainly analog filters as prototypes, which 
turn out to make simpler the expressions of the derived frequency mappings, and therefore 
the complexity of the designed 2D filters is lower in the analyzed cases. The prototypes used 
here were both maximally-flat or very selective, either low-pass or band-pass. For each type 
of 2D filter, a particular spectral transformation is derived. An important advantage is that 
these spectral transformations include some parameters which depend on the 2D filter 
specifications, like bandwidth, orientation, aperture etc. Once found the specific frequency 
mapping, the 2D filter results from its factorized prototype function by a simple substitution 
in each factor. The designed filters are versatile in the sense that the prototype parameters 
(bandwidth, selectivity) can be adjusted and the 2D filter will inherit these properties. 


An advantage of the analytical approach over the completely numerical optimization techni- 
ques is the possibility to control the 2D filter parameters by adjusting the prototype. Another 
novelty is the proposed analytical design method in polar coordinates, which can yield 
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selective two-directional and even multi-directional filters, and also fan and diamond filters. 
In polar coordinates more general filters with a specified rotation angle can be synthesized. 


The design methods approached here are rather simple, efficient and flexible, since by starting 
from different specifications, the matrices of a new 2D filter result directly by applying the 
determined frequency mapping, and there is no need to resume every time the whole design 
procedure. 


Stability of the designed filters is also an important problem and will be studied in detail in 
future work on this topic. In principle the spectral transformations used preserve the stability 
of the 1D prototype. The derived 2D filter could become unstable only if the numerical 
approximations introduce large errors. In this case the precision of approximation has to be 
increased by considering higher order terms, which would increase in turn the filter complex- 
ity; however, this is the price paid for obtaining efficient and stable 2D filters. Further research 
will focus on an efficient implementation of the designed filters and also on their applications 
in real-life image processing. 
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